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In the case of baseband signaling, the received waveforms are already in a pulse¬ 
like form. One might ask, why then, is a demodulator needed to recover the pulse 
waveforms? The answer is that the arriving baseband pulses are not in the form of 
ideal pulse shapes, each one occupying its own symbol interval. The filtering at the 
transmitter and the channel typically cause the received pulse sequence to suffer 
from intersymbol interference (ISI) and thus appear as an amorphous “smeared" 
signal, not quite ready for sampling and detection. The goal of the demodulator 
(receiving filter) is to recover a baseband pulse with the best possible signal-to- 
noise ration (SNR), free of any ISI. Equalization, covered in this chapter, is a 
technique used to help accomplish this goal. The equalization process is not 
required for every type of communication channel. However, since equalization 
embodies a sophisticated set of signal-processing techniques, making it possible to 

compensate for channel-induced interference, it is an important area for many 
systems. 

The bandpass model of the detection process, covered in Chapter 4, is virtu¬ 
ally identical to the baseband model considered in this chapter. That is because a 
received bandpass waveform is first transformed to a baseband waveform before 
the final detection step takes place. For linear systems, the mathematics of de¬ 
tection is unaffected by a shift in frequency. In fact, we can define an equivalence 
theorem as follows: Performing bandpass linear signal processing followed by 
heterodyning the signal to baseband, yields the same results as heterodyning the 
bandpass signal to baseband, followed by baseband linear signal processing. The 
term “heterodyning” refers to a frequency conversion or mixing process that yields 
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a spectral shift in the signal. As a result of this equivalence theorem, all linear 
signal-processing simulations can take place at baseband (which is preferred for 
simplicity) with the same results as at bandpass. This means that the performance 
of most digital communication systems will often be described and analyzed as if 
the transmission channel is a baseband channel. 


3.1 SIGNALS AND NOISE 

3.1.1 Error-Performance Degradation in Communication Systems 

The task of the detector is to retrieve the bit stream from the received waveform, 
as error free as possible, notwithstanding the impairments to which the signal may 
have been subjected. There are two primary causes for error-performance degrada¬ 
tion. The first is the effect of filtering at the transmitter, channel, and receiver, dis¬ 
cussed in Section 3.3, below. As described there, a nonideal system transfer 
function causes symbol “smearing” or intersymbol interference (ISI). 

Another cause for error-performance degradation is electrical noise and in¬ 
terference produced by a variety of sources, such as galaxy and atmospheric noise, 
switching transients, intermodulation noise, as well as interfering signals from other 
sources. (These are discussed in Chapter 5.) With proper precautions, much of the 
noise and interference entering a receiver can be reduced in intensity or even elim¬ 
inated. However, there is one noise source that cannot be eliminated, and that is 
the noise caused by the thermal motion of electrons in any conducting media. This 
motion produces thermal noise in amplifiers and circuits, and corrupts the signal in 
an additive fashion. The statistics of thermal noise have been developed using 
quantum mechanics, and are well known [1]. 

The primary statistical characteristic of thermal noise is that the noise ampli¬ 
tudes are distributed according to a normal or Gaussian distribution, discussed in 
Section 1.5.5, and shown in Figure 1.7. In this figure, it can be seen that the most 
probable noise amplitudes are those with small positive or negative values. In 
theory, the noise can be infinitely large, but very large noise amplitudes are rare. 
The primary spectral characteristic of thermal noise in communication systems, is 
that its two-sided power spectral density G„(f) = NJ2 is flat for all frequencies of 
interest. In other words, the thermal noise, on the average, has just as much power 
per hertz in low-frequency fluctuations as in high-frequency fluctuations—up to a 
frequency of about 10'^ hertz. When the noise power is characterized by such a 
constant-power spectral density, we refer to it as white noise. Since thermal noise is 
present in all communication systems and is the predominant noise source for 
many systems, the thermal noise characteristics (additive, white, and Gaussian, 
giving rise to the name AWGN) are most often used to model the noise in the 
detection process and in the design of receivers. Whenever a channel is designated 
as an AWGN channel (with no other impairments specified), we are in effect being 
told that its impairments are limited to the degradation caused by this unavoidable 
thermal noise. 
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3.1.2 Demodulation and Detection 


During a given signaling interval T, a binary baseband system will transmit one of 
two waveforms, denoted gi(/) and giit). Similarly, a binary bandpass system will 
transmit one of two waveforms, denoted A'i(r) and S 2 {t). Since the general treatment 
of demodulation and detection are essentially the same for baseband and bandpass 
systems, we use s^t) here as a generic designation for a transmitted waveform, 
whether the system is baseband or bandpass. This allows much of the baseband 
demodulation/detection treatment in this chapter to be consistent with similar 
bandpass descriptions in Chapter 4. Then, for any binary channel, the transmitted 
signal over a symbol interval (0, T) is represented by 

/ \ _ i(0 0 < r ^ r for a binary I 

\5 2(0 < r for a binary 0 

The received signal r(/) degraded by noise n(r) and possibly degraded by the 
impulse response of the channel was described in Equation (1.1) and is re¬ 
written as 

'■(0 =-^/(0 * *c(0 + ''(0 ' = 1. M (3.1) 

where n(t) is here assumed to be a zero mean AWGN process, and * represents a 
convolution operation. For binary transmission over an ideal distortionless channel 
where convolution with produces no degradation (since for the ideal case h^it) 
is an impulse function), the representation of r(f) can be simplified to 

r{() =Siit) + n{t) / = 1,2, 0<r < T (3.2) 

Figure 3.1 shows the typical demodulation and detection functions of a digital 
receiver. Some authors use the terms “demodulation” and “detection” inter¬ 
changeably. This book makes a distinction between the two. We define demodula¬ 
tion as recovery of a waveform (to an undistorted baseband pulse), and we 
designate detection to mean the decision-making process of selecting the digital 
meaning of that waveform. If error-correction coding not present, the detector out¬ 
put consists of estimates of message symbols (or bits), nti (also called hard deci¬ 
sions). If error-correction coding is used, the detector output consists of estimates 
of channel symbols (or coded bits) iJ,, which can take the form of hard or soft deci¬ 
sions (see Section 7.3.2). For brevity, the term “detection” is occasionally used 
loosely to encompass all the receiver signal-processing steps through the decision 
making step. The frequency down-conversion block, shown in the demodulator 
portion of Figure 3.1, performs frequency translation for bandpass signals operat¬ 
ing at some radio frequency (RF). This function may be configured in a variety of 
ways. It may lake place within the front end of the receiver, within the demodula¬ 
tor, shared between the two locations, or not at all. 

Within the demodulate and sample block of Figure 3.1 is the receiving filter 
(essentially the demodulator), which performs waveform recovery in preparation 
for the next important step—detection. The filtering at the transmitter and the 
channel typically cause the received pulse sequence to suffer from ISI, and thus it is 
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0,(7’) + rto(T) 


Figure 3.1 Two basic steps in the demodulation/detection of digital signals. 


not quite ready for sampling and detection. The goal of the receiving filter is to re¬ 
cover a baseband pulse with the best possible signal-to-noise ratio (SNR), free of 
any ISI. The optimum receiving filter for accomplishing this is called a matched 
filter or correlator, described in Sections 3.2.2 and 3.2.3. An optional equalizing 
filter follows the receiving filter; it is only needed for those systems where channel- 
induced ISI can distort the signals. The receiving filter and equalizing filter are 
shown as two separate blocks in order to emphasize their separate functions. In 
most cases, however, when an equalizer is used, a single filter would be designed to 
incorporate both functions and thereby compensate for the distortion caused by 
both the transmitter and the channel. Such a composite filter is sometimes referred 
to simply as the equalizing filter or the receiving and equalizing filter. 

Figure 3.1 highlights two steps in the demodulation/dctection process. Step 1, 
the waveform-to-sample transformation, is made up of the demodulator followed 
by a sampler. At the end of each symbol duration T, the output of the sampler, the 
predetection point, yields a sample z(T), sometimes called the test statistic. z{T) 
has a voltage value directly proportional to the energy of the received symbol and 
inversely proportional to the noise. In step 2, a decision (detection) is made regard¬ 
ing the digital meaning of that sample. We assume that the input noise is a Gaus¬ 
sian random process and that the receiving filter in the demodulator is linear. A 
linear operation performed on a Gaussian random process will produce a second 
Gaussian random process [2]. Thus, the filter output noise is Gaussian. The output 
of step 1 yields the test statistic 

Z{T) = a,{T) + n,{T) i = ],2 (3.3) 
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where (ijiT) is the desired signal component, and no(T) is the noise component. To 
simplify the notation, we sometimes express Equation (3.3) in the form of z = n, + 
/io. The noise component is a zero mean Gaussian random variable, and thus 
z{T) is a Gaussian random variable with a mean of either or «2 depending on 
whether a binary one or binary zero was sent. As described in Section 1.5.5, the 
probability density function (pdf) of the Gaussian random noise can be ex¬ 
pressed as 




1 


^“0 


Vz^ 


exp 


2 




(3.4) 


where o-q >s the noise variance. Thus it follows from Equations (3.3) and (3.4) that 
the conditional pdfs p(zIai) and p(zIa 2 ) can be expressed as 


P(2l^l) = 


1 




y/ZiT 


exp 


if 

2 V ao 


(3.5) 


and 


P(z|^2) = 




These conditional pdfs are illustrated in Figure 3.2. The rightmost conditional pdf, 
p(z\sx), called the likelihood of Si, illustrates the probability density function of the 
random variable zi T), given that symbol was transmitted. Similarly, the leftmost 
conditional pdf, p{z\s 2 ), called the likelihood of ^ 2 , illustrates the pdf of ziT), given 
that symbol S 2 was transmitted. The abscissa, z(T), represents the full range of pos¬ 
sible sample output values from step 1 of Figure 3.1. 

After a received waveform has been transformed to a sample, the actual 
shape of the waveform is no longer important; all waveform types that are trans¬ 
formed to the same value of z(T) are identical for detection purposes. Later it is 
shown that an optimum receiving filter (matched filter) in step 1 of Figure 3.1 maps 
all signals of equal energy into the same point z(r). Therefore, the received signal 
energy (not its shape) is the important parameter in the detection process. This is 
why the detection analysis for baseband signals is the same as that for bandpass sig- 


Likelihood of S 2 
p(z\s2) 


Likelihood of si 

p^z\s^) 



ziT) 


02 


Za(T) Oi 


Yo 


Figure 3.2 Conditional probability density functions: p(zlsi) and ^(zlsg). 
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nals. Since z{T) is a voltage signal that is proportional to the energy of the received 
symbol, the larger the magnitude of z{T), the more error free will be the decision¬ 
making process. In step 2, detection is performed by choosing the hypothesis that 
results from the threshold measurement 


Z{T) 2 (3.7) 

where and H 2 are the two possible (binary) hypotheses. The inequality relation¬ 
ship indicates that hypothesis //| is chosen if z{T) > y, and hypothesis Hi is chosen 
if z(7’) < 7. If ^(7") = 7, the decision can be an arbitrary one. Choosing Hi is equiva¬ 
lent to deciding that signal Si{() was sent and hence a binary 1 is detected. Similarly, 
choosing Hi is equivalent to deciding that signal ^ 2(0 was sent, and hence a binary 0 
is detected. 

3.1.3 A Vector View of Signals and Noise 

We now present a geometric or vector view of signal waveforms that are useful for 
either baseband or bandpass signals. We define an A^-dimensional orthogonal space 
as a space characterized by a set of N linearly independent functions {<})y(0K called 
basis fimetions. Any arbitrary function in the space can be generated by a lin¬ 
ear combination of these basis functions. The basis functions must satisfy the 
conditions 



where the operator 


0<r < r y,A: = 



for j = k 
otherwise 


(3.8a) 


(3.8b) 


is called the Kronecker delta function and is defined by Equation (3.8b). When the 
Kj constants are nonzero, the signal space is called orthogonal. When the basis 
functions are normalized so that each Kj - \, the space is called an orthonormal 
space. The principal requirement for orthogonality can be stated as follows. Each 
\\tj{t) function of the set of basis functions must be independent of the other mem¬ 
bers of the set. Each i]iy(r) must not interfere with any other members of the set in 
the detection process. From a geometric point of view, each i|iy(/) is mutually 
perpendicular to each of the other \\tf,{t) for j ^ k. An example of such a space with 
A^ = 3 is shown in Figure 3.3, where the mutually perpendicular axes are designated 
4 ^ 2 ( 0 * and If 4'/(0 corresponds to a real-valued voltage or current 

waveform component, associated with a 1-11 resistive load, then using Equations 
(1.5) and (3.8), the normalized energy in joules dissipated in the load in T seconds, 
due to i|jy, is 
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Figure 3.3 Vectorial representa¬ 
tion of the signal waveform Sm(/). 


V|/2U) 




One reason we focus on an orthogonal signal space is that Euclidean distance 
measurements, fundamental to the detection process, are easily formulated in 
such a space. However, even if the signaling waveforms do not make up such an 
orthogonal set, they can be transformed into linear combinations of orthogonal 
waveforms. It can be shown [3] that any arbitrary finite set of waveforms 
(/ = 1.A/), where each member of the set is physically realizable and of dura¬ 

tion 7. can be expressed as a linear combination of N orthogonal waveforms i|i|(/), 
i|i2(0» • • •» where N <M, such that 


.v,(r) = + ••• + 

•'' 2(0 = "2l‘l'l(0 + «224'2{0 + ••• + 


•'■..w(0 = + fl,W2‘l<2(0 + 


« ♦ 


+ a ) 


These relationships are expressed in more compact notation as 


•^/(O = 2 ^y (0 ^‘ = 1. ^ 

N^M 


where 


(3.10) 





/ = 1 . M O^t 

i = 


(3.11) 


The coefficient a^^ is the value of the i|yy(r) component of signal 5,(/). The form of the 
|i|jy(/)} is not specified; it is chosen for convenience and will depend on the form of 
the signal waveforms. The set of signal waveforms, {^/(r)), can be viewed as a set of 
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vectors, |sj = {«,], ... , example, /V = 3, we may plot the vector s„, 

corresponding to the waveform 


+ ^'m2‘1^2(0 + «,«3>l'3(0 

as a point in a three-dimensional Euclidean space with coordinates 
as shown in Figure 3.3. The orientation among the signal vectors describes the rela¬ 
tion of the signals to one another (with respect to phase or frequency), and the am¬ 
plitude of each vector in the set (s,) is a measure of the signal energy transmitted 
during a symbol duration. In general, once a set of orthogonal functions has been 
adopted, each of the transmitted signal waveforms, s^{t), is completely determined 
bv the vector of its coefficients, 

s, = ^ ^ “ 1,..., A/ (3.12) 

We shall employ the notation of signal vectors, |s}, or signal waveforms, |5(f)), 
as best suits the discussion. A typical detection problem, conveniently viewed in 
terms of signal vectors, is illustrated in Figure 3.4. Vectors Sjcmd s* represent proto¬ 
type or reference signals belonging to the set of M waveforms, (A',(t)|. The receiver 
knows, a priori, the location in the signal space of each prototype vector belonging to 
the A/-ary set. During the transmission of any signal, the signal is perturbed by noise 
so that the resultant vector that is actually received is a perturbed version (e.g., + n 
or n) of the original one, where n represents a noise vector. The noise is additive 
and has a Gaussian distribution; therefore, the resulting distribution of possible re¬ 
ceived signals is a cluster or cloud of points around s^ and s^. The cluster is dense in the 
center and becomes sparse with increasing distance from the prototype. The arrow 
marked 'T” represents a signal vector that might arrive at the receiver during some 
symbol interval. The task of the receiver is to decide whether r has a close “resem- 





Figure 3.4 Signals and noise in a 
three-dimensional vector space. 
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blance" to the prototype s^, whether it more closely resembles or whether it is 
closer to some other prototype signal in the M-ary set. The measurement can be 
thought of as a distance measurement. The receiver or detector must decide which of 
the prototypes within the signal space is closest in distance to the received vector r. 
The analysis of all demodulation or detection schemes involves this concept of 
distance between a received waveform and a set of possible transmitted waveforms. 
A simple rule for the detector to follow' is to decide that r belongs to the same class as 
its nearest neighbor (nearest prototype vector). 

3.1.3.1 Waveform Energy 

Using Equations (1.5), (3.10), and (3.8), the normalized energy associated 
with the waveform Si{t) over a symbol interval T can be expressed in terms of the 
orthogonal components of Si(t) as follows: 


£- = f sj{t)(it = r [ df 

J 0 J „ L ' J 

(3.13) 

= r 2 Oij >i<;(o 2 ii.Mo dt 
"^0 ^ 

(3.14) 

= 22 a,jaik 1 
f Jq 

(3.15) 


(3.16) 

II 

• 

II 

• 

• 

(3.17) 

Equation (3.17) is a special case of ParsevaFs theorem relating the integral of the 
square of the waveform 5j(/) to the sum of the square of the orthogonal series coef¬ 
ficients. If orthonormal functions are used (i.e.. A',— 1), the normalized energy over 
a symbol duration T is given by 

II 

(3.18) 

If there is equal energy E in each of the s,(t) waveforms, we can 
(3.18) in the form 

write Equation 

.V 

E = uJi for all i 

(3.19) 


/-I 

3.1.3.2 Generalized Fourier Transforms 

The transformation described by Equations (3.8), (3.10), and (3.11) is re¬ 
ferred to as the generalized Fourier transformation. In the case of ordinary Fourier 
transforms, the (v^/O} set comprises sine and cosine harmonic functions. But in the 
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case of generalized Fourier transforms, the |4^;(/)) set is not constrained to any spe¬ 
cific form; it must only satisfy the orthogonality statement of Equation (3.8). Any 
arbitrary integrable waveform set, as well as noise, can be represented as a linear 
combination of orthogonal waveforms through such a generalized Fourier transfor¬ 
mation [3]. Therefore, in such an orthogonal space, we are justified in using dis¬ 
tance (Euclidean distance) as a decision criterion for the detection of any signal set 
in the presence of AWGN. The most important application of this orthogonal 
transformation has to do with the w'ay in which signals are actually transmitted and 
received. The transmission of a nonorthogonal signal set is generally accomplished 
by the appropriate weighting of the orthogonal carrier components. 

Example 3.1 Orthogonal Representation of Waveforms 

Figure 3.5 illustrates the statement that any arbitrary integrable waveform set can be 

represented as a linear combination of orthogonal waveforms. Figure 3.5a shows a set 

of three waveforms, 5|(/), .V 2 (/), and ^^(O* 

(a) Demonstrate that these waveforms do not form an orthogonal set. 

(b) Figure 3.5b shows a set of two waveforms. v|i|(A) and ili2(0- Verify that these wave¬ 
forms form an orthogonal set. 

(e) Show how the nonorthogonal waveform set in part (a) can he expressed as a linear 
combination of the orthogonal set in part (b). 

(d) Figure 3.5c illustrates another orthogonal set of two waveforms, iM(0 and *1/2(0- 
Show how the nonorthogonal set in Figure 3.5a can be expressed as a linear com¬ 
bination of the set in Figure 3.5c. 

Solution 

(a) .v,(0, S 2 {t). and 53(0 are clearly not orthogonal, since they do not meet the 
requirements of Equation (3.8); that is, the time integrated value (over a symbol 
duration) of the cross-product of any two of the three waveforms is not zero. Let 
us verify this for 5|(r) and 52 ( 0 : 

r A’i(/)5-2(o = r Si{t)s2{t)d (r S](t)s2{r)df 

T/2 

I 

= r {-\){2)di + r (-3)(o)rff = -T 

J 0 J yy2 

Similarly, the integral over the interval T of each of the cross-products S](t)sj^(t) 
and S 2 {t)Sy{t) results in nonzero values. Hence, the waveform set (5,(01 U = F 2. 3) 
in Figure 3.5a is not an orthogonal set. 

(b) Using Equation (3.8), we verify that *|j,(0 and *li2(0 form an orthogonal set as 
follows: 




{l){l)dt + 



(c) Using Equation (3.11) with Kj = T, we can express the nonorthogonal set |5j(/)) (/ = 
1,2,3) as a linear combination of the orthogonal basis waveforms |*|i/01 (/ = L 2): 
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szit) 






t 






T 

f y\fj{t)^\^k{t)dt = 
•'0 


Tforj = k 
0 otherwise 





T 


rfor7 = ^ 

0 otherwise 



Figure 3.5 Example of an arbitrary signal set in terms of an orthogonal set. 
(a) Arbitrary signal set. (b) A set of orthogonal basis functions, (c) Another set of 
orthogonal basis functions. 


■Sl(0 = >l'l(0 - 2>1/2(/) 

•«2(0 = + wo 

J,l(0 = - 1|J2(0 

(d) Similar to part (c), the nonorthogonal set (i,(/)l (/ = 1, 2. 3) can be expressed in 
terms of the simple orthogonal basis set (»|<y(/)| (/ = •. 2) in Figure 3.5c, as follows: 

*i(0 = -»l'i’(0 - 3W0 

4'2(0 = 2>ii;(/) 

^3(0 = m - 3W0 

These relationships illustrate how an arbitrary waveform set ( 5 :,(01 can be ex¬ 
pressed as a linear combination of an orthogonal set (WOl- as described in Equa¬ 
tions (3.10) and (3.11). What are the practical applications of being able to 
describe 5|(r), S 2 (t), and ^^(t) in terms of >|/|(0. »p 2 ( 0 - and the appropriate coeffi- 
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dents? If we want a system for transmitting waveforms 5i{0, ^2(0’ 53 ( 0 , the 

transmitter and the receiver need only be implemented using the two basis func¬ 
tions and i|/ 2 (/) instead of the three original waveforms. The Gram-Schmidt 
orthogonalization procedure provides a convenient way in which an appropriate 
choice of a basis function set can be obtained for any given signal set jyXOl- 

(It is described in Appendix 4A of Reference [4].) 


3.1,3*3 Representing White Noise with Orthogonal Waveforms 

Additive white Gaussian noise (AWGN) can be expressed as a linear combi¬ 
nation of orthogonal waveforms in the same way as signals. For the signal detection 
problem, the noise can be partitioned into two components, 

n{t) = h{() -f h{t) (3.20) 

where 

n{t) = 2 (3.21) 

;=i 

is taken to be the noise within the signal space, or the projection of the noise 

components on the signal coordinates i|ii(0» • • • 1 <ind 

h{t) = n{t) - fi{t) (3.22) 

is defined as the noise outside the signal space. In other words, h{t) may be thought 
of as the noise that is effectively tuned out by the detector. The symbol n{t) 
represents the noise that will interfere with the detection process. We can express 
the noise waveform n{t) as 

n{t) = 2 + «(0 (3.23) 

/ = ! 

where 


and 





dt = Q for all j 


(3.25) 


The interfering portion of the noise, h(t), expressed in Equation (3.21) will hence¬ 
forth be referred to simply as n{t). We can express n(t) by a vector of its coefficients 
similar to the way we did for signals in Equation (3.12). We have 


n = (/Zi,Ai 2 , ...,A/^) (3.26) 

where n is a random vector with zero mean and Gaussian distribution, and where 
the noise components (i = 1. N) are independent. 
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3.1.3.4 Variance of White Noise 


White noise is an idealized process with two-sided power spectral density 
equal to a constant for all frequencies from -ac to +ac. Hence, the noise 
variance (average noise power, since the noise has zero mean) is 


a* = var 




(3.27) 


Although the variance for AWGN is infinite, the variance for filtered A WON 
is finite. For example, if AWGN is correlated with one of a set of orthonormal 
functions the variance of the correlator output is given by 



= var (n^) = E 





(3.28) 


The proof of Equation (3.28) is given in Appendix C. Henceforth we shall assume 
that the noise of interest in the detection process is the output noise of a correlator 
or matched filter with variance = Ao/2 as expressed in Equation (3.28). 


3.1.4 The Basic SNR Parameter for Digital Communication Systems 


Anyone who has studied analog communications is familiar with the figure of 
merit, average signal power to average noise power ratio (S/N or SNR). In digital 
communications, we more often use a normalized version of SNR, as a 

figure of merit. is bit energy and can be described as signal power S times the bit 
time Tfj. is noise power spectral density, and can be described as noise power N 
divided by bandwidth W. Since the bit time and bit rate are reciprocal, we can 
replace Tf, with 1//?^ and write 


E,^ST,^ S/R, 
N/W N/W 


(3.29) 


Data rate, in units of bits per second, is one of the most recurring parameters in 
digital communications. We therefore simplify the notation throughout the book, 
by using R instead of Rf, to represent bits/s, and we rewrite Equation (3.29) to 
emphasize that is just a version of S/N normalized by bandwidth and bit rate, 
as follows: 


Ao “ A V £ y 


(3.30) 


One of the most important metrics of performance in digital communication sys¬ 
tems is a plot of the bit-error probability versus £^/A(). Figure 3.6 illustrates the 
“waterfall-like” shape of most such curves. For £/,/A(, >.Vo, £« < Pn- The dimension¬ 
less ratio £/,/A,) is a standard quality measure for digital communications system 
performance. Therefore, required EJN^ can be considered a metric that character¬ 
izes the performance of one system versus another; the smaller the required E^N^, 
the more efficient is the detection process for a given probability of error. 
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For E}ylNQ > xo, Pb ^ 

Nq n[r 


xo 


E^/Nq 


Figure 3.6 General shape of the Pb 
versus E^/N^ curve. 


3.1.5 Why EJN^ Is a Natural Figure of Merit 

A newcomer to digital communications may question the usefulness of the parame¬ 
ter £/,/Ao* After all, S/N is a useful figure of merit for analog communications—the 
numerator represents a power measurement of the signal we wish to preserve and 
deliver, and the denominator represents electrical noise degradation. Moreover, 
S/N is intuitively acceptable as a metric of goodness. Thus, why can’t we continue 
to use S/N as a figure of merit for digital communications? Why has a different 
metric for digital systems—the ratio of bit energy to noise power spectral density- 
arisen? The explanation is given below. 

In Section 1.2.4, a power signal was defined as a signal having finite average 
power and infinite energy. An energy signal was defined as a signal having zero av¬ 
erage power and finite energy. These classifications are useful in comparing analog 
and digital waveforms. We classify an analog waveform as a power signal. Why 
does this make sense? We can think of an analog waveform as having infinite dura¬ 
tion that need not be partitioned or windowed in time. An infinitely long electrical 
waveform has an infinite amount of energy; hence, energy is not a useful way 
to characterize this waveform. Power (or rate of delivering the energy) is a more 
useful parameter for analog waveforms. 

However, in a digital communication system we transmit (and receive) a sym¬ 
bol by using a transmission waveform within a window of time, the symbol time 
Focusing on one symbol, we can see that the power (averaged over all time) goes to 
zero. Hence, power is not a useful way to characterize a digital waveform. What we 
need for such waveforms is a metric of the “good stuff” within the window. In other 
words, the symbol energy (power integrated over T^) is a more useful parameter 
for characterizing digital waveforms. 
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The fact that a digital signal is best characterized by its received energy 
doesn't yet get to the crux of why is a natural metric for digital systems, so let 
us continue. The digital waveform is a vehicle that represents a digital message. 

The message may contain one bit (binary), two bits (4-ary).10 bits (1024-ary). 

In analog systems, there is nothing akin to such a discretized message structure. An 
analog information source is an infinitely quantized continuous wave. For digital 
systems, a figure of merit should allow us to compare one system with another at 
the bit level. Therefore, a description of the digital waveform in terms of S/N is vir¬ 
tually useless, since the waveform may have a one-bit meaning, a two-bit meaning, 
or a 10-bit meaning. For example, suppose we are told that for a given error proba¬ 
bility, the required S/N for a digital binary waveform is 20 units. Think of the wave¬ 
form as being interchangeable with its meaning. Since the binary waveform has a 
one-bit meaning, then the S/N requirement per bit is equal to the same 20 units. 
However, suppose that the waveform is 1024-ary, with the same 20 units of re¬ 
quired S/N, Now, since the waveform has a 10-bit meaning, the S/N requirement 
per bit is only 2 units. Why should we have to go through such computational ma¬ 
nipulations to find a metric that represents a figure of merit? Why not immediately 
describe the metric in terms of what we need—an energy-related parameter at the 
bit level, Ei^/NqI Just as S/N is a dimensionless ratio, so too is EJN^, To verify this, 
consider the following units of measure: 

_ Joule _ Watt-s 
Ai, Watt per Hz Watt-s 


3.2 DETECTION OF BINARY SIGNALS IN GAUSSIAN NOISE 

3.2.1 Maximum Likelihood Receiver Structure 

The decision-making criterion shown in step 2 of Figure 3.1 was described by 
Equation (3.7) as 



A popular criterion for choosing the threshold level 7 for the binary decision in 
Equation (3.7) is based on minimizing the probability of error. The computation 
for this minimum error value of 7 = 70 starts with forming an inequality expression 
between the ratio of conditional probability density functions and the signal a pri¬ 
ori probabilities. Since the conditional density function p{z\Si) is also called the like¬ 
lihood of Si, the formulation 


P{Z\S^) ^ P{S2) 

Piz\s2) P{Si) 


(3.31) 
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is called the likelihood ratio test. (See Appendix B.) In this inequality, f*( 5 ,) and 
P{S 2 ) are the a priori probabilities that .y,(/) and S 2 {t), respectively, are transmitted, 
and Hi and H 2 are the two possible hypotheses. The rule for minimizing the error 
probability states that we should choose hypothesis Hi if the ratio of likelihoods is 
greater than the ratio of a priori probabilities, as shown in Equation (3.31). 

It is shown in Section B.3.1, that if /’(s,) = P{s 2 ). and if the likelihoods, 
pizlSj) (/ = 1,2), are symmetrical, the substitution of Equations (3.5) and (3.6) into 
(3.31) yields 




^1 + (ii 
2 



(3.32) 


where Oi is the signal component of ziT) when SiU) is transmitted, and 02 is the 
signal component of z(T) when 5,(0 is transmitted. The threshold level 7 ,,- repre¬ 
sented by (fl| + 02 ) 12 , is the optimum threshold for minimizing the probability of 
making an incorrect decision for this important special case. This strategy is known 
as the minimum error criterion. 

For equally likely signals, the optimum threshold 70 passes through the inter¬ 
section of the likelihood functions, as shown in Figure 3.2. Thus by following Equa¬ 
tion (3.32), the decision stage effectively selects the hypothesis that corresponds to 
the signal with the maximum likelihood. For example, given an arbitrary detector 
output value z„{T), for which there is a nonzero likelihood that Za{T) belongs to 
either signal class 5 |(/) or i,( 0 , one can think of the likelihood test as a comparison 
of the likelihood values p(zjsi) and p{zjs 2 ). The signal corresponding to the maxi¬ 
mum pdf is chosen as the most likely to have been transmitted. In other words, the 
detector chooses ,Vi(t) if 


> P(zji’2) (3.33) 

Otherwise, the detector chooses .V 2 ( 0 - ^ detector that minimizes the error prob¬ 
ability (for the case where the signal classes are equally likely) is also known as a 
maximum likelihood defector. 

Figure 3.2 illustrates that Equation (3.33) is just a “common sense" way to 
make a decision when there exists statistical knowledge of the classes. Given the 
detector output value ZaiT). we see in Figure 3.2 that ZaiT) intersects the likeli¬ 
hood of Slit) at a value €|, and it intersects the likelihood of ^ 2(0 a value Hj. What 
is the most reasonable decision for the detector to make? For this example, choos¬ 
ing class 5 i(/), which has the greater likelihood, is the most sensible choice. If this 
was an M-ary instead of a binary example, there would be a total of M likelihood 
functions representing the M signal classes to which a received signal might belong. 
The maximum likelihood decision would then be to choose the class that had the 
greatest likelihood of all M likelihoods. (Refer to Appendix B for a review of deci¬ 
sion theory fundamentals.) 
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3.2.1.1 Error Probability 

For the binary decision-making depicted in Figure 3.2, there are two ways 
errors can occur. An error e will occur when Si(t) is sent, and channel noise results 
in the receiver output signal z{/) being less than 70 . The probability of such an oc¬ 
currence is 


/ •/<> 

/j(z|.v,) ciz 

- X 


(3.34) 


This is illustrated by the shaded area to the left of 70 in Figure 3.2. Similarly, an 
error occurs when ^ 2(0 is sent, and the channel noise results in z{T) being greater 
than 7 (). The probability of this occurrence is 


P{e\s2) = P{H, 


l‘^2) = f P{z\ 


52 ) dz 


(3.35) 


The probability of an error is the sum of the probabilities of all the ways that an 
error can occur. For the binary case, we can express the probability of bit error as 


Pb= 'Z = Z Pie\si)P{si) 

(=l 1=1 

Combining Equations (3.34) to (3.36), we can write 


(3.36) 


Pa= P(e\si)P{si) + P{e\s2)P{s2) 


(3.37a) 


or equivalently. 


Pb= P{H2\Sl)P{Si) ^ PmS2)P{S2) 


(3.37b) 


That is, given that signal .Vt(/) was transmitted, an error results if hypothesis H 2 
is chosen; or given that signal 52(0 was transmitted, an error results if hypothesis 
//j is chosen. For the case where the a priori probabilities are equal [that is. 


^(^ 1 ) - ^('^' 2 )- 2 ]’ 


P, = '2P{H2\s,)+-2P{PllU2) 


(3.38) 


and because of the symmetry of the probability density functions, 

P,i = P{H2 \s,) = P{H^S2) (3.39) 


The probability of a bit error. Pg, is numerically equal to the area under the “tail” 
of either likelihood function, p(zl 5 i) or p(z\s 2 ), falling on the “incorrect” side of the 
threshold. We can therefore compute Pg by integrating pizlsi) between the limits 
-» and 7 o> or by integrating p{zls 2 ) between the limits 70 and 



(3.40) 


Here, 7 n = (fl| +a2)/2 is the optimum threshold from Equation (3.32). Replacing the 
likelihood p(z\s 2 ) with its Gaussian equivalent from Equation (3.6), we have 
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21 


(3.41) 




1 


7o = (« 


<.)/2 ^ 


exp 



where (Jq is the variance of the noise out of the correlator. 
Let » = (z - Then oq du = dz and 







du = Q 



(3.42) 


where Q(x), called the complementary error function or co-error function, is a 
commonly used symbol for the probability under the tail of the Gaussian pdf. It is 
defined as 



Note that the co-error function is defined in several ways (see Appendix B); how¬ 
ever, all definitions are equally useful for determining probability of error in 
Gaussian noise. Q{x) cannot be evaluated in closed form. It is presented in tabular 
form in Table B.i. Good approximations to Q(x) by simpler functions can be found 
in Reference [5]. One such approximation, valid for x > 3, is 



(3.44) 


We have optimized (in the sense of minimizing Pg) the threshold level y, but 
have not optimized the receiving filter in block 1 of Figure 3.1. We next consider 
optimizing this filter by maximizing the argument of Q{x) in Equation (3.42). 


3.2.2 The Matched Filter 


A matched filter is a linear filter designed to provide the maximum signal-to-noise 
power ratio at its output for a given transmitted symbol waveform. Consider that a 
known signal 5 (/) plus AWGN n(t) is the input to a linear, time-invariant (receiv¬ 
ing) filter followed by a sampler, as shown in Figure 3.1. At time t= T, the sampler 
output z(7’) consists of a signal component Oj and a noise component no- The 
variance of the output noise (average noise power) is denoted by 0 - 5 , so that the 
ratio of the instantaneous signal power to average noise power, iS/N)j, at time 
t = r, out of the sampler in step 1 , is 



(3.45) 


We wish to find the filter transfer function that maximizes Equation (3.45). 
We can express the signal aft) at the filter output in terms of the filter transfer 
function H(f) (before optimization) and the Fourier transform of the input 
signal, as 
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(3.46) 


a ,(0 = 



X 


H{f)S{f)e>^'’f'df 


X 


where S(f) is the Fourier transform of the input signal, s(t). If the two-sided power 
spectral density of the input noise is Nq/2 watts/hertz, then, using Equations (1.19) 
and (1.53), we can express the output noise power as 


= 





K 




(3.47) 


— K 


We then combine Equations (3.45) to (3.47) to express (S/N)t, as follows; 


N,r 



X 


//(/)5(/)e''^rf/ 


— X 


X 


(3.48) 


No/2 


H{f)\^df 


^ — X 


We next find that value of H(f) = Ho(f) for which the maximum {S/N)t\s achieved, 
by using Schwarz's inequality. One form of the inequality can be stated as 



X 


.fi{x)f2{x) dx 


— X 



X 


^ X 


\f^{x)\^dx f \f 2 ix)\^dx 

J ^x 


(3.49) 


The equality holds if/i(A:) = kf^ix), where k is an arbitrary constant and ♦ indicates 
complex conjugate. If we identify H(f) with /,(x) and S(f) with f 2 {x), we can 
write 



OC 


H{f)S{f)e 


X 


X 


\Hif)\^df 


— X 



X 


\S{f)\^df 


(3.50) 


X 


Substituting into Equation (3.48) yields 


N 


T NoJ_^ 


IS(f)i^df 


(3.51) 


or 


2 £ 

max 1 — = — 

N/t No 


(3.52) 


where the energy E of the input signal s(t) is 


-r 

• X 


IS(f)l^df 


(3.53) 


Thus, the maximum output (S/AOr depends on the input signal energy and the 
power spectral density of the noise, not on the particular shape of the waveform 
that is used. 

The equality in Equation (3.52) holds only if the optimum filter transfer func¬ 
tion Ha(f) is employed, such that 
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H{f) = Hif) = 


(3.54) 



h{t) = 


(3.55) 


Since .y(/) is a real-valued signal, we can write, from Equations ( A.29) and (A.31), 



ks{T- t) 
0 


0 < r < r 

elsewhere 


(3.56) 


Thus, the impulse response of a filter that produces the maximum output signal-lo- 
noise ratio is the mirror image of the message signal 5(t), delayed by the symbol 
time duration T. Note that the delay of T seconds makes Equation (3.56) causal; 
that is, the delay of T seconds makes h{t) a function of positive time in the interval 
0 < t < T, Without the delay of T seconds, the response s (-() is unrealizable 
because it describes a response as a function of negative time. 


3.2.3 Correlation Realization of the Matched Filter 


Equation (3.56) and Figure 3.7a illustrate the matched filter's basic property: The 
impulse response of the filter is a delayed version of the mirror image (rotated on 
the ( = 0 axis) of the signal waveform. Therefore, if the signal waveform is 5(t), its 
mirror image is s(-t), and the mirror image delayed by T seconds is s{T - t). The 
output z{t) of a causal filter can be described in the time domain as the convolution 
of a received input waveform r{t) with the impulse response of the filter (see Sec¬ 
tion A.5): 


z(r) = r{() * h{t) = r r{7)h{t - t) di (3.57) 

"'0 

Substituting h(t) of Equation (3.56) into h(( - t) of Equation (3.57) and arbitrarily 
setting the constant k equal to unity, we get 



r(T)5[r — (t — t)] dj 


r{r)s{T — t + t) dr 


When / = 7, we can write Equation (3.58) as 


(3.58) 


z{T) = r r(T)5(T)dT (3.59) 

0 * 

The operation of Equation (3.59), the product integration of the received signal r{t) 
with a replica of the transmitted waveform s(t) over one symbol interval is known 
as the correlation of r(t) with s(t). Consider that a received signal r(t) is correlated 
with each prototype signal 5/(/) (/ = 1, ... , M), using a bank of M correlators. The 
signal Si(t) whose product integration or correlation with r{t) yields the maximum 
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si-t) 


hit) = s(T—t) 


T 

Signal waveform 
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signal waveform 




T 
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Matched filter 
output 


t 



Figure 3.7 Correlator and matched filter, (a) Matched filter characteris¬ 
tic. (b) Comparison of correlator and matched filter outputs. 


output Zi{T) is the signal that matches r(t) better than all the other 5y(r), j ^ i. We 
will subsequently use this correlation characteristic for the optimum detection of 
signals. 

3.2.3.1 Comparison of Convolution and Correlation 

The mathematical operation of a matched filter (MF) is convolution; a signal 
is convolved with the impulse response of a filter. The mathematical operation of a 
correlator is correlation; a signal is correlated with a replica of itself. TTie term 
“matched filter" is often used synonymously with “correlator." How is that possi¬ 
ble when their mathematical operations are different? Recall that the process of 
convolving two signals reverses one of them in time. Also, the impulse response of 
an MF is defined in terms of a signal that is reversed in time. Thus, convolution in 
the MF with a time-reversed function results in a second time-reversal, making the 
output (at the end of a symbol time) appear to be the result of a signal that has 
been correlated with its replica. Therefore, it is valid to implement the receiving fil¬ 
ter in Figure 3.1 with either a matched filter or a correlator. It is important to note 
that the correlator output and the matched filter output are the same only at time 
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t=T. For a sine-wave input, the output of the correlator, z(0, is approximately a 
linear ramp during the interval 0 < r < T. However, the matched filter output is ap¬ 
proximately a sine wave that is amplitude modulated by a linear ramp during the 
same time interval. The comparison is shown in Figure 3.7b. Since for comparable 
imputs, the MF output and the correlator output are identical at the sampling time 
i - 1\ the matched filter and correlator functions pictured in Figure 3.8 are often 
used interchangeably. 


3.2.3.2 Dilemma in Representing Earliest versus Latest Event 

A serious dilemma exists in representing timed events. This dilemma is un¬ 
doubtedly the cause of a frequently made error in electrical engineering—confus- 

% 

ing the most significant bit (MSB) with the least significant bit (LSB). Figure 3.9a 
illustrates how a function of time is typically plotted; the earliest event appears left¬ 
most, and the latest event rightmost. In western societies, where we read from left 
to right, would there be any other way to plot timed events? Consider Figure 3.9b, 
where pulses are shown entering (and leaving) a network or circuit. Here, the 
earliest events are shown rightmost, and the latest are leftmost. From the figure, it 
should be clear that whenever we denote timed events, there is an inference that 
we are following one of the two formats described here. Often, we have to 
provide some descriptive words (e.g., the rightmost bit is the earliest bit) to avoid 
confusion. 

Mathematical relationships often have built-in features guaranteeing the 
proper alignment of time events. For example, in Section 3.2.3, a matched filter is 
defined as having an impulse response h(i) that is a delayed version of the lime- 
reversed copy of the signal. That is, h(t) = s{T - t). Delay of one symbol time T is 
needed for the filter to be causal (the output must occur in positive time). Time re¬ 
versal can be thought of as a ‘'precorrection” where the rightmost part of the time 
plot will now correspond to the earliest event. Since convolution dictates another 
time reversal, the arriving signal and the filter’s impulse response will be “in step” 
(earliest with earliest, and latest with latest). 


r(t) = Siit) + nit) 



ziT) 


Matched to 

S^it)-S2(t) 



r(t) = Siit) + nit) 




ziT) 


Figure 3.8 Equivalence of matched 
filter and correlator, (a) Matched fil¬ 
ter. (b) Correlator. 
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Figure 3.9 Dilemma in representing 
earliest versus latest events. 
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3.2.4 Optimizing Error Performance 


To optimize (minimize) in the context of an AWGN channel and the receiver 
shown in Figure 3.1, we need to select the optimum receiving filter in step 1 and the 
optimum decision threshold in step 2. For the binary case, the optimum decision 
threshold has already been chosen in Equation (3.32), and it was shown in Equa¬ 
tion (3,42) that this threshold results in Fg = 0 [{a^ — r?2)/2cTQj. Next, for minimizing 
Pg, it is necessary to choose the filter (matched filter) that maximizes the argument 
of Q(x), Thus, we need to determine the linear filter that maximizes (o^ - 
or equivalently, that maximizes 



(3.60) 


where (r/j - ai) is the difference of the desired signal components at the filter out¬ 
put at time / = T, and the square of this difference signal is the instantaneous power 
of the difference signal. In Section 3.2.2, a matched filter was described as one that 
maximizes the output signal-to-noisc ratio (SNR) for a given known signal. Here, 
we continue that development for binary signaling, where we view the optimum 
filter as one that maximizes the difference between two possible signal outputs. 
Starting with Equations (3.45) and (3.47), it was shown in Equation (3.52) that a 
matched filter achieves the maximum possible output SNR equal to 2E/Nq. 
Consider that the filter is matched to the input difference signal - .V2(0]; thus, 
we can write an output SNR at lime r= Tas 





(3.61) 


where Nq/2 is the two-sided power spectral density of the noise at the filter input 
and 
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(3.62) 






is the energy of the difference signal at the filter input. Note that Equation (3.61) 
does not represent the SNR for any single transmission, s^{t) or S 2 (t). This SNR 
yields a metric of signal difference for the filter output. By maximizing the output 
SNR as shown in Equation (3.61), the matched filter provides the maximum 
distance (normalized by noise) between the two candidate outputs—signal n, and 
signal ^ 2 - 

Next, combining Equations (3.42) and (3.61) yields 



For the matched filter, Equation (3.63) is an important interim result in terms 
of the energy of the difference signal at the filter's input. From this equation, a 
more general relationship in terms of received bit energy can be developed. We 
start by defining a time cross-correlation coefficient p as a measure of similarity be¬ 
tween two signals, 5|(r) and 52(0- We have 

p = ^ r Slit) S 2 {t) dt (3.64a) 

and 

p = cos 0 (3.64b) 

where -1 < p < 1. Equation (3.64a) is the classical mathematical way of expressing 
correlation. However, when 5i(0 and .^2(0 are viewed as signal vectors, S| and S 2 , re¬ 
spectively, then p is conveniently expressed by Equation (3.64b). This vector view 
provides a useful image. The vectors Sj and S 2 are separated by the angle 6; for 
small angular separation, the vectors are quite similar (highly correlated) to each 
other, and for large angular separation, they are quite dissimilar. The cosine of this 
angle gives us the same normalized metric of correlation as Equation (3.64a). 
Expanding Equation (3.62), we get 

Ed= i s\{t)dt + r S2{t)dt-2 r s^it)s 2 {t)dt (3.65) 

-'o -^0 

Recall that each of the first two terms in Equation (3.65) represents the energy as¬ 
sociated with a bit, that is, 

= r >^ 1 ( 0 ^^ = r (3.66) 

-'0 -^0 

Substituting Equations (3.64a) and (3.66) into Equation (3.65), we get 

2p£, = 2£,(1 - p) (3.67) 

Substituting Equation (3.67) into (3.63), we obtain 
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Pb^Q 


£.(1 - p) 


(3.68) 



A^o 


Consider the case of p = 1 corresponding to signals 5i(f) and ^2(0 being perfectly 
correlated over a symbol time (drawn as vectors, with the angle between them 
equal to zero). Would anyone use such waveforms for digital signaling? Of course 
not, because the communication signals (members of the alphabet set) need to be 
as disparate from one another as possible so that they are easily distinguished (de¬ 
tected). We are simply cataloging the possible values for the parameter p. Consider 
the case of p = -1 corresponding to Sx{t) and S 2 it) being “anticorrelated” over a 
symbol time. In other words, the angle between the signal vectors is 180"^. In such a 
case, where the vectors are mirror images, we call the signals antipodal, as shown in 
Figure 3.10a. Also, consider the case of p = 0 corresponding to zero correlation 
between s^{t) and i’2(0 (ihe angle between the vectors is 90"). We call such signals 
orthogonal, as seen in Figure 3.10b. For two waveforms to be orthogonal, they must 
be uncorrelated over a symbol interval; that is, 




(3.69) 


The subject of orthogonality was treated earlier in Section 3.1.3. For the case of de¬ 
tecting antipodal signals (that is, p = -l) with a matched filter, Equation (3.68) can 
be written as 





V |/2 



V” 

<Eb 


Figure 3.10 Binary signal vectors, (a) Antipodal. 

(b) Orthogonal. (b) 
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Similarly, for the case of detecting orthogonal signals (that is, p = 0) with a matched 
filter. Equation (3.68) can be written as 

/■.-e(Vl) (3.71) 

Figure 3.10, where the signal magnitudes are each shown equal to V^, helps 
illustrate that the error performance described by Equations (3.70) and (3.71) is a 
function of the distance between Sj and sh (the larger the distance, the smaller will 
be Fg). When the signals are antipodal, as in Figure 3.10a, the distance between 
them is 2 V^, and the energy E^l associated with that distance is characterized by 
the distance squared or 4£^. When we substitute E,i = into Equation (3.63), the 
result is Equation (3.70). W hen the signals are orthogonal as in Figure 3.10b, the 
distance between them is vlE^ and thus Ej = 2 E^. When we substitute £,/-2 Ef, 
into Equation (3.63), the result is Equation (3.71). 

Example 3.2 Matched Filter Detection of Antipodal Signals 

Consider a binary communications system that receives equally likely signals s^it) and 
S 2 (t) plus AWGN. See Figure 3.11. Assume that the receiving filter is a matched filter 
(MF), and that the noise-power spectral density Ay is equal to 10 Watl/Hz. Use the 
values of received signal voltage and lime shown on Figure 3.11 to compute the bit- 
error probability. 

Solution 

We can graphically determine the received energy per bit from the plot of either 5|(r) 
or ^ 2(0 shown in Figure 3.11 by integrating to find the energy (area under the voltage- 
squared pulse). Doing this in piecewise fashion, we get 



= (10‘'W)2 X (lO’^'s) + (2 X 10-^V)2 X (10"S) + (10"*'V)- X (10 '’s) 
= 6 X 10*^^ joule 


si(^) 

(millivolts) 



0 12 3 


t (ps) 


S2{t) 



t (ps) 


Figure 3.11 Baseband antipodal waveforms. 
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Since the waveforms depicted in Figure 3.11 are antipodal and are delected with a 
matched filter, we use Equation (3.70) to find the bit-error probability, as 

= Qi^) = C>(3.46) 

From Table B.l, we find that 10 ^ Or, since the argument of Q{x) is greater 

than 3, we can also use the approximate relationship in Equation (3.44) which yields 
Ffl-2.9x H)-^. 

Because the received signals are antipodal and are received by an MF, these are 
sulficient prerequisites such that Equation (3.70) provides the proper relationship for 
finding bit-error probability. The waveforms .Vi(/) and .V 2 (/) could have been pictured in 
a much more bizarre fashion, but as long as they are antipodal and detected by an MF, 
their shapes do not enter into the Pft computations. The shapes of the waveforms, of 
course, do matter when it comes to specifying the impulse response of the MF needed 
to detect these waveforms. 


3.2.5 Error Probability Performance of Binary Signaling 


3.2.5.1 Unipolar Signaling 

Figure 3.12a illustrates an example of baseband orthogonal signaling 
namely, unipolar signaling, where 


s |(/) = A 0 < r < 7* for binary 1 

■^' 2(0 — ^ Q ^ t ^ T for binary 0 


(3.72) 


and where .4 > 0 is the amplitude of symbol s^{t). The definition of orthogonal sig¬ 
naling described by Equation (3.69) requires that s^{t) and 82 ( 1 ) have zero correla¬ 
tion over each symbol time duration. Because in Equation (3.72), S 2 (() is equal to 
zero during the symbol time, this set of unipolar pulses clearly fulfills the condition 



Figure 3.12 Detection of unipolar 
baseband' signaling, (a) Unipolar 
signaling example, (b) Correlator 
detector. 


Reference signal 

si(f)-S2(^) = A 
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shown in Equation (3.69), and hence, they form an orthogonal signal set. Consider 
such unipolar signaling, as illustrated in Figure 3.12a, as well as the correlator 
shown in Figure 3.12b, which can be used for detecting such pulses. The correlator 
multiplies and integrates the incoming signal r{t) with the difference of the proto¬ 
type signals, - .riCO] = After a symbol duration T, a sampler (inferred by 
the upper limit of integration) yields the test statistic z(7'), which is then compared 
with the threshold For the case of 5,(0 plus AWGN being received—that is. 
when r{t) = 5 ,(/) + n {{)—the signal component of ziT) is found, using Equation 
(3.59), to be 


fl,(7’) = E{z(r)|si(0} = E|J A^ + An{t)dt'j = A^T 

where E {z(7')IS|(/)) is the expected value of 2(7), given that i|(/) was sent. This 
follows since E (rt(r)l = 0. Similarly, when r(r) = S 2 (t) + n(r), then «2(7) = 0. Thus, 
in this case, the optimum decision threshold, from Equation (3.32), is given by 
-yu = (aI + a2)/2 = 1/2 A^T. If the test statistic z(T) is greater than y„, the signal is 
declared to be .V|(/); otherwise, it is declared to be s^iO- 

The energy difference signal, from Equation (3.62), is given by Ej = A^T. 
Then the bit-error performance at the output is obtained from Equation (3.63) as 



where, for the case of equally likely signaling, the average energy per bit is = 
A^TI2. Equation (3.73) corroborates Equation (3.71) where this relationship was 
established for orthogonal signaling in a more general way. 

Note that the units out of a multiplier circuit, such as that seen in Figure 3.12b 
are volts. Therefore, for voltage signals on each of the two inputs, the multiplier 
transfer function must have units of 1/volt, and the measurable units of r{t) 5,(0 out 
of the multiplier are volt/volt-squared. Similarly, the units out of an integrator cir¬ 
cuit arc also volts. Thus, for a voltage signal into an integrator, the integrator trans¬ 
fer function must have units of 1/second, and thus the overall transfer function of 
the product integrator has units of 1/volt-sccond. Then, for a signal into the product 
integrator having units of volt-squarcd-seconds (a measure of energy), we 
represent the output z(7) as a voltage signal that is proportional to received signal 
energy (volt/joule). 


3.2.5.2 Bipolar Signaling 

Figure (3.13a) illustrates an example of baseband antipodal signaling— 
namely, bipolar signaling, where 

5 ,(f) = +A 0 ^ t T for binary 1 

■ and (3.74) 


■^ 2(0 = -A for binary 0 
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Si(t) 



Reference signal 
si(f) = A 



Figure 3.13 Detection of bipolar baseband signaling, 
(a) Bipolar signaling example, (b) Correlator detector. 


As defined earlier, the term “antipodal” refers to binary signals that are mirror im¬ 
ages of one another; that is, Ai(r) = - 52 (f)- A correlator receiver for such antipodal 
waveforms can be configured as shown in Figure 3.13b. One correlator multiplies 
and integrates the incoming signal r(t) with the prototype signal 5,(t); the second 
correlator multiplies and integrates r{t) with 52 (f). 

Figure 3.13b captures the essence of a digital receiver’s main function. That is, 
during each symbol interval, a noisy input signal is sent down multiple “avenues” in 
an effort to correlate it with each of the possible candidates. The receiver then 
seeks the largest output voltage (the best match) to make a detection. For the bi¬ 
nary example, there are just two possible candidates. For a 4-ary example, there 
would be four candidates, and so forth. In Figure 3.13b, the correlator outputs are 
designated Zi{T) (/ = 1, 2). The test statistic, formed from the difference of the cor¬ 
relator outputs, is 

z{T) = Zi{T) - Z 2 iT) (3.75) 

and the decision is made using the threshold shown in Equation (3.32). For antipo¬ 
dal signals, a^ = - 02 , therefore, -y„ = 0. Thus, if the test statistic z{T) is positive, the 
signal is declared to be 5i(T); and if it is negative, it is declared to be 52 ( 7 ’). 
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From Equation (3.62), the energy-difference signal is = {lAfT. Then, the 
bit-error performance at the output can be obtained from Equation (3.63) as 








(3.76) 


where the average energy per bit is given by E^j = A^T. Equation (3.76) corrobo¬ 
rates Equation (3.70) where this relationship was established for antipodal signal¬ 
ing in a more general way. 


3.2,5.3 Signaling Described with Basis Functions 

Instead of designating Si(t) as the reference signals in the correlator of Figure 
3.13b, we can use the concept of basis functions described in Section 3.1.3. Binary 
signaling with unipolar or bipolar pulses provides particularly simple examples for 
doing this, because the entire signaling space can be described by just one basis 
function. If we normalize the space by choosing /Cy = 1 in Eq uatio n (3.9), then it 
should be clear that the basis function must be equal to Vl/ T- 
For unipolar pulse signaling, we could then write 

i-i(f) = = aVt X = A 


and 


■^2(0 “ — 0 ^ 




where the coefficients and equaM'S/r and 0, respectively. 
For bipolar pulse signaling, we would write 



and 


S2{t) = a2i4'i(0 = -aVt X 

where the coefficients and ^21 equal A\/Y and -A\/t, respectively. For the 
case of antipodal pulses, we can envision the corr elato r receiver taking the form of 
Figure 3.12b with the reference signal equal to VT/T- Then, for the case of ^i(f) = 
A being sent, we can write 






This follows because E {n(f)} = 0, and since, for antipodal signaling, Ej, = A^T, it fol¬ 
lows that ai(T) = Vf*- Similarly, for a received signal r(f) = 52 (f) - 1 - n{t), it follows 
that a 2 {T) = -V^. When reference signals are treated in this way, then the ex- 
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pected value of z(T) has a magnitude of V^, which has units of normalized volts 
proportional to received energy. This basis-function treatment of the correlator 
yields a convenient value of z(T) that is consistent with units of volts out of multi¬ 
pliers and integrators. We therefore repeat an important point: At the output of 
the sampler (the predetection point), the test statistic z{T) is a voltage signal that is 
proportional to received signal energy. 

Figure 3.14 illustrates curves of versus E^INq for bipolar and unipolar sig¬ 
naling. There are only two fair ways to compare such curves. By drawing a vertical 
line at some given f/yAy, say, 10 dB, we see that the unipolar signaling yields in 
the order of 10"\ but that the bipolar signaling yields Pq in the order of 10"^. The 
lower curve is the better performing one. Also, by drawing a horizontal line at 
some required Pg, say 10*^, we see that with unipolar signaling each received bit 
would require an of about 12.5 dB, but with bipolar signaling, we could get 
away with requiring each received bit to have an E^IN^ of only about 9.5 dB. Of 
course, the smaller requirement of £‘,,//V„ is better (using less power, smaller batter¬ 
ies). In general, the better performing curves are the ones closest to the axes, lower 
and leftmost. In examining the two curves in Figure 3.14, we can see a 3-dB error- 
performance improvement for bipolar compared with unipolar signaling. This 



Figure 3.14 Bit error performance of unipolar and bipolar signaling. 
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difference could have been predicted by the factor-of-2 difference in the coefficient 
of £(,/No 't> Equation (3.70) compared with (3.71). In Chapter 4, it is shown that, 
with MF detection, bandpass antipodal signaling (e.g., binary phase-shift keying) 
has the same Pg performance as baseband antipodal signaling (e.g., bipolar pulses). 
It is also shown that, with MF detection, bandpass orthogonal signaling (e.g., or¬ 
thogonal frequency-shift keying) has the same Pg performance as baseband orthog¬ 
onal signaling (e.g., unipolar pulses). 


3.3 INTERSYMBOL INTERFERENCE 

Figure 3.15a introduces the filtering aspects of a typical digital communication sys¬ 
tem. There are various filters (and reactive circuit elements such as inductors and 
capacitors) throughout the system—in the transmitter, in the receiver, and in the 
channel. At the transmitter, the information symbols, characterized as impulses or 
voltage levels, modulate pulses that are then filtered to comply with some band¬ 
width constraint. For baseband systems, the channel (a cable) has distributed reac¬ 
tances that distort the pulses. Some bandpass systems, such as wireless systems, are 
characterized by fading channels (see Chapter 15), that behave like undesirable fil¬ 
ters manifesting signal distortion. When the receiving filter is configured to com¬ 
pensate for the distortion caused by both the transmitter and the channel, it is often 
referred to as an equalizing filter or a receiving/equalizing filter. Figure 3.15b illus¬ 
trates a convenient model for the system, lumping all the filtering effects into one 
overall equivalent system transfer function 

H{f) = Hff) H,{f) Hff) (3.77) 


a:i X 2 
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Figure 3.15 Intersymbol interference in the detection process, (a) Typical baseband 
digital system, (b) Equivalent model. 
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where //,(/) characterizes the transmitting filter, //,.(/) the filtering within the chan¬ 
nel, and H,{f) the receiving/equalizing filter. The characteristic //(/), then, repre¬ 
sents the composite system transfer function due to all the filtering at various 
locations throughout the Iransmitter/channel/receiver chain. In a binary system 
with a common PCM waveform, such as NRZ-L, the detector makes a symbol deci¬ 
sion by comparing a sample of the received pulse to a threshold; for example, the 
detector in Figure 3.15 decides that a binary one was sent if the received pulse is 
positive, and that a binary zero was sent, if the received pulse is negative. Due to 
the effects of system filtering, the received pulses can overlap one another as 
shown in Figure 3.15b. The tail of a pulse can “smear” into adjacent symbol inter¬ 
vals, thereby interfering with the detection process and degrading the error perfor¬ 
mance; such interference is termed intersymhol interference (ISI). Even in the 
absence of noise, the effects of filtering and channel-induced distortion lead to ISI. 
Sometimes H^f) is specified, and the problem remains to determine //,(/) and 
such that the ISI is minimized at the output of 

Nyquist [6] investigated the problem of specifying a received pulse shape so 
that no ISI occurs at the detector. He showed that the theoretical minimum system 
bandwidth needed in order to detect /?, symbols/s, without ISI, is RJ2 hertz. This 
occurs when the system transfer function H{f) is made rectangular, as shown in 
Figure 3.16a. For baseband systems, when H{f) is such a filter with single-sided 
bandwidth 1/27’ (the ideal Nyquist filter), its impulse response, the inverse Fourier 
transform of H{f) (from Table A. 1) is of the form h{t) = sine (t/T), shown in Figure 
3.16b. This sine (t/T)-shaped pulse is called the ideal Nyquist pulse; its multiple 
lobes comprise a main lobe and sidelobes called pre- and post-mainlobe tails that 
are infinitely long. Nyquist established that if each pulse of a received sequence is 
of the form sine (t/T), the pulses can be detected without ISI. Figure 3.16b illus¬ 
trates how ISI is avoided. There are two successive pulses, h{t) and h{t - T). Even 
though /*(/) has long tails, the figure shows a tail passing through zero amplitude at 
the instant (r = T) when h {t - T) is to be sampled, and likewise all tails pass 
through zero amplitude when any other pulse of the sequence h{t - kT), k = ±\, 
±2, ... is to be sampled. Fherefore, assuming that the sample timing is perfect, 
there will be no ISI degradation introduced. For baseband systems, the bandwidth 


mn h(t) 
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(a) (b) 

Figure 3.16 Nyquist channels for zero ISI. (a) Rectangular system 
transfer function H{f). (b) Received pulse shape h{t) = sine {t/T). 
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required to detect MT such pulses (symbols) per second is equal to 1/27"; in other 
words, a system with bandwidth W = MIT = RJl hertz can support a maximum 
transmission rate of 2V^ = MT = symbols/s {Nyquist bandwidth constraint) with¬ 
out ISI. Thus, for ideal Nyquist filtering (and zero ISI), the maximum possible 
symbol transmission rate per hertz, called the symbol-rate packings is 
2 symbols/s/Hz. It should be clear from the rectangular-shaped transfer function of 
the ideal Nyquist filter and the infinite length of its corresponding pulse, that such 
ideal filters are not realizable; they can only be approximately realized. 

The names “Nyquist filter” and “Nyquist pulse” are often used to describe 
the general class of filtering and pulse-shaping that satisfy zero ISI at the sampling 
points. A Nyquist filter is one whose frequency transfer function can be repre¬ 
sented by a rectangular function convolved with any real even-symmetric fre¬ 
quency function. A Nyquist pulse is one whose shape can be represented by a sine 
(t/T) function multiplied by another time function. Hence, there are a countless 
number of Nyquist filters and corresponding pulse shapes. Amongst the class of 
Nyquist filters, the most popular ones are the raised cosine and the root-raised 
cosine, treated below. 

A fundamental parameter for communication systems is bandwidth efficiency, 
R/W, whose units are bits/s/Hz. As the units imply, RAV represents a measure of 
data throughput per hertz of bandwidth and thus measures how efficiently any 
signaling technique utilizes the bandwidth resource. Since the Nyquist bandwidth 
constraint dictates that the theoretical maximum symbol-rate packing without ISI 
is 2 symbols/s/Hz, one might ask what it says about the maximum number of 
bits/s/Hz. It says nothing about bits, directly; the constraint deals only with pulses 
or symbols, and the ability to detect their amplitude values without distortion from 
other pulses. To find RAV for any signaling scheme, one must know how many bits 
each symbol represents, which is a separate issue. Consider an A/-ary PAM signal¬ 
ing set. Each symbol (comprising k bits) is represented by one of A/-pusle ampli¬ 
tudes. For k = 6 bits per symbol, the symbol set size is M = 2* = 64 amplitudes. Thus 
with 64-ary PAM, the theoretical maximum bandwidth efficiency that is possible 
without ISI is 12 bits/s/Hz. (Bandwidth efficiency is treated in greater detail in 
Chapter 9.) 

3.3.1 Pulse Shaping to Reduce ISI 

3.3.1.1 Goals and Trade-offs 

The more compact we make the signaling spectrum, the higher is the allow¬ 
able data rate or the greater is the number of users that can simultaneously be 
served. This has important implications to communication service providers, since 
greater utilization of the available bandwidth translates into greater revenue. For 
most communication systems (with the exception of spread-spectrum systems, cov¬ 
ered in Chapter 12), our goal is to reduce the required system bandwidth as much 
as possible. Nyquist has provided us with a basic limitation to such bandwidth re¬ 
duction. What would happen if we tried to force a system to operate at smaller 
bandwidths than the constraint dictates? The pulses would become spread in time. 
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which would degrade the system's error performance due to increased ISl. A pru¬ 
dent goal is to compress the bandwidth of the data impulses to some reasonably 
small bandwidth greater than the Nyquist minimum. This is accomplished by pulse¬ 
shaping with a Nyquist filter. If the band edge of the filter is sleep, approaching the 
rectangle in Figure 3.16a, then the signaling spectrum can be made most compact. 
However, such a filter has an impulse response duration approaching infinity, as in¬ 
dicated in Figure 3.16b. Each pulse extends into every pulse in the entire sequence. 
Long time responses exhibit large-amplitude tails nearest the main lobe of each 
pulse. Such tails are undesirable because, as shown in Figure 3.16b, they contribute 
zero ISl only when the sampling is performed at exactly the correct sampling time; 
when the tails are large, small timing errors will result in ISl. Therefore, although a 
compact spectrum provides optimum bandwidth utilization, it is very susceptible to 
ISl degradation induced by timing errors. 


3,3.L2 The Raised-Cosine Filter 

Earlier, it was stated that the receiving filter is often referred to as an equaliz¬ 
ing filter, when it is configured to compensate for the distortion caused by both the 
transmitter and the channel. In other words, the configuration of this filter is 
chosen so as to optimize the composite system frequency transfer function //(/), 
shown in Equation (3.77). One frequently used H{f) transfer function belonging to 
the Nyquist class (zero ISl at the sampling times) is called the raised-cosine filter. It 
can be expressed as 




/tt I/I +W-2W^, \ 

V 4 VF - VTo ; 


for I/I < 2VTo - W 1 
for 2Wq- W < \f\ < w \ (3.78) 
for \f\ > W ) 


where W is the absolute bandwidth and Wq = 1/2T represents the minimum Nyquist 
bandwidth for the rectangular spectrum and the -6-dB bandwidth (or half- 
amplitude point) for the raised-cosine spectrum. The difference W - W^^is termed 
the “excess bandwidth,” which means additional bandwidth beyond the Nyquist 
minimum (i.e., for the rectangular spectrum, W is equal to Wq). The roll-off factor 
is defined to be r = (W - Wq)IWq, where 0 < r < 1. It represents the excess band¬ 
width divided by the filter -6-dB bandwidth (i.e., the fractional excess bandwidth). 
For a given Wq, the roll-off r specifies the required excess bandwidth as a fraction 
of Wq and characterizes the steepness of the filter roll off. The raised-cosine charac¬ 
teristic is illustrated in Figure 3.17a for roll-off values of /* = 0, r = 0.5, and /•= 1. The 
/* = 0 roll-off is the Nyquist minimum-bandwidth case. Note that when r = 1, the 
required excess bandwidth is 100%, and the tails are quite small. A system with 
such an overall spectral characteristic can provide a symbol rate of symbols/s 
using a bandwidth of hertz (twice the Nyquist minimum bandwidth), thus 
yielding a symbol-rate packing of 1 symbol/s/Hz. The corresponding impulse 
response for the H(f) of Equation (3.78) is 
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Figure 3.17 Raised-cosine filter characteristics, (a) System transfer 
function, (b) System impulse response. 


h{t) = IWoisinc 2Wot) 


cos [2tt{W - Wo)r] 
1 - [4{W - Wo)tY 


(3.79) 


and is plotted in Figure 3.17b for r = 0, r = 0.5, and r = 1. The tails have zero value at 
each pulse-sampling time, regardless of the roll-off value. 

We can only approximately implement a filter described by Equation (3.78) 
and a pulse shape described by Equation (3.79), since, strictly speaking, the raised- 
cosine spectrum is not physically realizable (for the same reason that the ideal 
Nyquist filter is not realizable). A realizable filter must have an impulse response 
of finite duration and exhibit a zero output prior to the pulse turn-on time (see Sec- 
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tion 1.7.2), which is not the case for the family of raised-cosine characteristics. 
These unrealizable filters are noncausal (the filter impulse response has infinite 
duration, and the filtered pulse begins at time / = -x). A pulse-shaping filter should 
satisfy two requirements. It should provide the desired roll-off, and it should be 
realizable (the impulse response needs to be truncated to a finite length). 

Starting with the Nyquist bandwidth constraint that the minimum required 
system bandwidth W for a symbol rale of symbols/s without ISI is RJ2 hertz, a 
more general relationship between required bandwidth and symbol transmission 
rate involves the filter roll-off factor r and can be stated as 

W = (3.80) 

Thus, with r = 0, Equation (3.80) describes the minimum required bandwidth for 
ideal Nyquist filtering. For r > 0, there is a bandwidth expansion beyond the 
Nyquist minimum; thus, for this case, R^ is now less than twice the bandwidth. If the 
demodulator outputs one sample per symbol, then the Nyquist sampling theorem 
has been violated, since we are left with too few samples to reconstruct the analog 
waveform unambiguously (aliasing is present). However, for digital communica¬ 
tion systems, we are not interested in reconstructing the analog waveform. Since 
the family of raised-cosine filters is characterized by zero ISI at the times that the 
symbols are sampled, we can still achieve unambiguous detection. 

Bandpass-modulated signals (see Chapter 4), such as amplitude shift keying 
(ASK) and phase-shift keying (PSK), require twice the transmission bandwidth of 
the equivalent baseband signals. (See Section 1.7.1.) Such frequency-translated sig¬ 
nals, occupying twice their baseband bandwidth, are often called double-sideband 
(DSB) signals. Therefore, for ASK- and PSK-modulatcd signals, the relationship 
between the required DSB bandwidth VP^sb ^^d the symbol transmission rate R^ is 

(1 + r)R, (3.81) 

Recall that the raised-cosine frequency transfer function describes the com¬ 
posite H{f) that is the ‘'full round trip'’ from the inception of the message (as an 
impulse) at the transmitter, through the channel, and through the receiving filter. 
The filtering at the receiver is the compensating portion of the overall transfer 
function to help bring about zero ISI with an overall transfer function, such as the 
raised cosine. Often this is accomplished by choosing (matching) the receiving filter 
and the transmitting filter so that each has a transfer function known as a root- 
raised cosine (square root of the raised cosine). Neglecting any channel-induced 
ISI, the product of these root-raised-cosine functions yields the composite raised- 
cosine system transfer function. Whenever a separate equalizing filter is introduced 
to mitigate the effects of channel-induced ISI, the receiving and equalizing filters 
together should be configured to compensate for the distortion caused by both 
the transmitter and the channel so as to yield an overall system transfer function 
characterized by zero ISI. 

Let’s review the trade-off that faces us in specifying pulse-shaping filters. The 
larger the filter roll-off, the shorter will be the pulse tails (which implies smaller tail 
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amplitudes). Small tails exhibit less sensitivity to timing errors and thus make for 
small degradation due to ISI. Notice in Figure 3.17b, for r = 1, that timing errors 
can still result in some ISI degradation. However, the problem is not as serious as it 
is for the case in which r = 0, because the tails of the /i(f) waveform are of much 
smaller amplitude for r = 1 than they are for r = 0. The cost is more excess 
bandwidth. On the other hand, the smaller the filter roll-off, the smaller will be the 
excess bandwidth, thereby allowing us to increase the signaling rate or the number 
of users that can simultaneously use the system. The cost is longer pulse tails, larger 
pulse amplitudes, and thus, greater sensitivity to timing errors. 


3.3.2 Two Types of Error-Performance Degradation 

The effects of error-performance degradation in digital communications can be 
partitioned into two categories. The first is due to a decrease in received signal 
power or an increase in noise or interference power, giving rise to a loss in signal- 
to-noise ratio or EJNq. The second is due to signal distortion, such as might be 
caused by intersymbol interference (ISI). Let us demonstrate how different are the 
effects of these two degradation types. 

Suppose that we require a communication system with a bit-error probability 
Ffl versus EJN^ characteristic corresponding to the solid-line curve plotted in Fig¬ 
ure 3.18a. Suppose that after the system is configured and measurements are taken, 
we find, to our disappointment, that the performance does not follow the theoreti¬ 
cal curve, but in fact follows the dashed line plot. A loss in Ef,INQ has come about 
because of some signal losses or an increased level of noise or interference. For a 




Figure 3.18 (a) Loss in EJNq. (b) Irreducible Pb caused by distortion. 
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desired bit-error probability of 10■^ the theoretical required E^/Nq is 10 dB. Since 
our system performance falls short of our goal, we can see from the dashed-line 
curve that, for the same bit-error probability of 10"\ the required EiJNq is now 12 
dB. If there were no way to remedy this problem, how much more EiJN^^ would 
have to be provided in order to meet the required bit-error probability? The an¬ 
swer is 2 dB, of course. It might be a serious problem—especially if the system is 
power-limited, and it is difficult to come up with the additional 2 dB. But that loss 
in Ef^/No is not so terrible when compared with the possible effects of degradation 
caused by a distortion mechanism. 


In Figure 3.18b, again imagine that we do not meet the desired performance 
of the solid-line curve. But instead of suffering a simple loss in signal-to-noise ratio, 
there is a degradation effect brought about by ISI (plotted with the dashed line). If 
there were no way to remedy this problem, how much more would be re¬ 

quired in order to meet the desired bit-error probability? It would require an infi¬ 
nite amount—or, in other words, there is no amount of E{^/Nq that will ameliorate 
this problem. More E^^INq cannot help when the curve manifests such an irreducible 
Pb (assuming that the boltoming-out point is located above the system’s required 
Pb)- Undoubtedly, every P^-versus-E^/No curve bottoms out somewhere, but if the 
bottoming-out point is well below the region of interest, it will be of no conse¬ 
quence. 

More Ef^/No may not help the ISI problem (it won’t help at all if the Pg curve 
has reached an irreducible level). This can be inferred by looking at the overlapped 
pulses in Figure 3.15b; if we increase the the ratio of that overlap does not 

change. The pulses are subject to the same distortion. What, then, is the usual cure 
for the degradation effects of ISI? The cure is found in a technique called equaliza¬ 
tion. (See Section 3.4.) Since the distortion effects of ISI are caused by filtering in 
the transmitter and the channel, equalization can be thought of as the process that 
reverses such nonoptimum filtering effects. 


Example 3.3 Bandwidth Requirements 

(a) Find the minimum required bandwidth for the baseband transmission of a four- 
level PAM pulse sequence having a data rate of P = 2400 bits/s if the system trans¬ 
fer characteristic consists of a raised-cosine spectrum with 100% excess bandwidth 
(r=l). 

(b) The same 4-ary PAM sequence is modulated onto a carrier wave, so that the base¬ 
band spectrum is shifted and centered at frequency^). Find the minimum required 
DSB bandwidth for transmitting the modulated PAM sequence. Assume that the 
system transfer characteristic is the same as in part (a). 

Soluiion 

(a) M = 2^; since M = 4 levels, k = 2. 


Symbol or pulse rate ^ = T = = 1200 symbols/s; 

Minimum bandwidth W = 4- r)R, = 2(2)(1200) = 1200Hz. 

Figure 3.19a illustrates the baseband PAM received pulse in the time domain—an 
approximation to the h(t) in Equation (3.79). Figure 3.19b illustrates the Fourier 
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Figure 3.19 (a) Shaped pulse, (b) Baseband raised cosine spectrum. 


transform of h{t )—the raised cosine spectrum. Notice that the required band¬ 
width, W, starts at zero frequency and extends to/= 1/7'; it is twice the size of the 
Nyquist theoretical minimum bandwidth. 

(b) As in part (a), 

= 1200 symbols/s; 

Wdsb = (I + r)R, = 2(1200) = 2400 Hz. 

Figure 3.20a illustrates the modulated PAM received pulse. This waveform can be 
viewed as the product of a high-frequency sinusoidal carrier wave and a waveform 
with the pulse shape of Figure 3.19a. The single-sided spectral plot in Figure 3.20b 
illustrates that the modulated bandwidth is 



When the spectrum of Figure 3.19b is shifted up in frequency, the negative and 
positive halves of the baseband spectrum are shifted up in frequency, thereby dou- 


h(t-tQ) 



to-T to to + T 


B(n 



WDSB = (fo + yT)-(fo-1/T) 

= 2/r 



Figure 3.20 (a) Modulated shaped pulse, (b) DSB-modulated raised cosine spec¬ 
trum. 
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bling the required transmission bandwidth. As the name implies, the DSB signal 
has two sidebands: the upper sideband (USB), derived from the baseband positive 
half, and the lower sideband (LSB), derived from the baseband negative half. 

Example 3.4 Digital Telephone Circuits 

Compare the system bandwidth requirements for a terrestrial 3-kHz analog telephone 
voice channel with that of a digital one. For the digital channel, the voice is formatted 
as a PCM bit stream, where the sampling rate for the analog-to-digital (A/D) conver¬ 
sion is 8000 samples/s and each voice sample is quantized to one of 256 levels. The bit 
stream is then transmitted using a PCM waveform and received with zero ISl. 

Solution 

The result of the sampling and quanilization process yields PCM words such that each 
word (representing one sample) has one of L = 256 different levels. If each sample 
were sent as a 256-ary PAM pulse (symbol), then from Equation (3.80) we can write 
that the required system bandwidth (without ISI) for sending symbols/s would be 

W ^ — hertz 
2 


where the equality sign holds true only for ideal Nyquist filtering. Since the digital 
telephone system uses PCM (binary) waveforms, each PCM word is converted to 
^ = log 2 L = log 2 256 = 8 bits. Therefore, the system bandwidth required to transmit 
voice using PCM is 


^PCM 


s (log2 L) 



hertz 



^ ^ (8 bits/symbol) (8000 symbols/s) = 32 kHz 


The 3-kHz analog voice channel will generally require approximately 4-kHz of band¬ 
width, including some bandwidth separation between channels, called guard bands. 
Therefore, the PCM format, using 8-bit quantization and binary signaling with a PCM 
waveform, requires at least eight times the bandwidth required for the analog channel. 


3.3.3 Demodulation/Detection of Shaped Pulses 

3.3.3.1 Matched Filters versus Conventional Filters 

Conventional filters screen out unwanted spectral components of a received 
signal while maintaining some measure of fidelity for signals occupying a selected 
span of the spectrum, called the pass-band. These filters are generally designed to 
provide approximately uniform gain, a linear phase-versus-frequency characteristic 
over the pass-band, and a specified minimum attenuation over the remaining spec¬ 
trum, called the stop-band(s). A matched filter has a different ‘‘design priority,” 
namely that of maximizing the SNR of a known signal in the presence of AWGN. 
Conventional filters are applied to random signals defined only by their bandwidth, 
while matched filters are applied to known signals with random parameters (such 
as amplitude and arrival time). The matched filter can be considered to be a tem¬ 
plate that is matched to the known shape of the signal being processed. A conven- 
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tional filter tries to preserve the temporal or spectral structure of the signal of in¬ 
terest. On the other hand, a matched filter significantly modifies the temporal 
structure by gathering the signal energy matched to its template, and, at the end of 
each symbol time, presenting the result as a peak amplitude. In general, a digital 
communications receiver processes received signals with both kinds of filters. The 
task of the conventional filter is to isolate and extract a high-fidelity estimate of the 
signal for presentation to the matched filter. The matched filter gathers the re¬ 
ceived signal energy, and when its output is sampled (at / = T), a voltage propor¬ 
tional to that energy is produced for subsequent detection and post-detection 
processing. 

3.3.3.2 Nyquist Pulse and Square-Root Nyquist Pulse 

Consider a sequence of data impulses at a transmitter input compared with 
the resulting sequence of pulses out of a raised-cosine matched filter (before sam¬ 
pling). In Figure 3.21, transmitted data is represented by impulse waveforms that 
occur at times to, T], ... Filtering spreads the input waveforms, and thus delays 
them in time. We use the notation, to, q,..., to denote received time. The impulse 
event that was transmitted at time Tq arrives at the receiver at time /(, corresponding 
to the start of the output pulse event. The premainlobe tail of a demodulated pulse 
is referred to as its precursor. For a real system with a fixed system-time reference, 
causality dictates that > t,,, and the time difference between Tq and ^ represents 
any propagation delay in the system. In this example, the time duration from the 
start of a demodulated pulse precursor until the appearance of its mainlobe or peak 
amplitude is 3T (three pulse-time durations). Each output pulse in the sequence is 
superimposed with other pulses; each pulse has an effect on the main lobes of three 
earlier and three later pulses. When a pulse is filtered (shaped) so that it occupies 
more than one symbol time, we define the pulse support time as the total number 
of symbol intervals over which the pulse persists. In Figure 3.21, the pulse support 
time consists of 6-symbol intervals (7 data points with 6 intervals between them). 

The impulse response of a root-raised cosine filter, called the sqiiare-root 
Nyquist pulse, is shown in Figure 3.22a (normalized to a peak value of unity, with a 
filter rolloff of r = 0.5). The impulse response of the raised-cosine fitter, called the 





Pulse support time 


Figure 3.21 Filtered impulse sequence: output versus input. 
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Nyquist pulse, is shown in Figure 3.22b (with the same normalization and filter 
rolloff). Inspecting these two pulse shapes, we see that they have a similar appear¬ 
ance, but the square-root Nyquist pulse makes slightly faster transitions, thus its 
spectrum (root-raised cosine) does not decay as rapidly as the spectrum (raised 
cosine) of the Nyquist pulse. Another subtle but important difference is that the 
square-root Nyquist pulse does not exhibit zero ISl (you can verify that the pulse 
tails in Figure 3.22a do not go through zero amplitude at the symbol times). 
However, if a root-raised cosine filter is used at both the transmitter and the 
receiver, the product of these transfer functions being a raised cosine, will give rise 
to an output having zero ISl. 

It is interesting to see how the square-root Nyquist pulses appear at the out¬ 
put of a transmitter and how they appear after demodulation with a root-raised co¬ 
sine MF. Figure 3.23a illustrates an example of sending a sequence of message 
symbols j+1 +1 -1 +3 +1 -i-3| from a 4-ary set, where the members of the alphabet 
set are: |±1, ±3). Consider that the pulse modulation is 4-ary PAM, and that the 
pulses have been shaped with a root-raised cosine filter, having a roll-off value of 
0.5. The analog waveform in this figure represents the transmitter output. Since the 
output waveform from any filter is delayed in time, then in Figure 3.23a, the input 
message symbols (shown as approximate impulses) have been delayed the same 
amount as the output waveform in order to align the message sequence with its cor¬ 
responding filtered waveform (the square-root Nyquist shaped-pulse sequence). 
This is just a visual convenience so that the reader can compare the filter input with 
its output. It is, of course, only the output analog waveform that is transmitted (or 
modulated) onto a carrier wave. 



Time 

Figure 3.22a Square-root Nyquist pulse. 
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Time 


Figure 3.22b Nyquist pulse. 

Figure 3.23b shows the same delayed message samples together with the out¬ 
put waveform from the root-raised cosine MF, yielding a raised-cosine transfer 
function for the overall system. Let us describe a simple test to determine if the 
filtered output (assuming no noise) contains ISI. It is only necessary to sample 
the filtered waveform at the times corresponding to the original input samples; if 



Figure 3.23a Square-root Nyquist-shaped M-ary waveform and 
delayed-input sample values. 


148 


Baseband Demodulation/Detection Chap. 3 





Time 


Figure 3.23b Output of raised-cosine matched filter and delayed-input 
sample values. 

the resulting sample values are unchanged from those of the original message, then 
the filter output has zero ISI (at the sample times). When Figures 3.23a and 3.23b 
are compared with regard to ISI, it should be apparent that sampling the square- 
root Nyquist waveform of Figure 3.23a (transmitter output) will not yield the exact 
original samples; however, sampling the Nyquist waveform in Figure 3.23b (MF 
output) will yield the exact original samples. This supports the statement that a 
Nyquist filter yields zero ISI at the sample points, while any other filter does not 
do so. 

3.4 EQUALIZATION 

3.4.1 Channel Characterization 

Many communication channels (e.g., telephone, wireless) can be characterized as 
band-limited linear filters with an impulse response h^it) and a frequency response 

/^(/) = (3-82) 

where hjj) and H^if) are Fourier transform pairs, \Hc(f)\ is the channel’s ampli¬ 
tude response, and 0^(/) is the channel’s phase response. In order to achieve ideal 
(nondistorting) transmission characteristics over a channel, it was shown in Section 
1.6.3, that within a signal’s bandwidth W, must be constant. Also, 0^(/) must 

he a linear function of frequency, which is tantamount to saying that the delay must 
be constant for all spectral components of the signal. If \Hc(f)\ is not constant 
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within W, then the effect is amplitude distortion. If is not a linear function of 
frequency within W, then the effect is phase distortion. For many channels that ex¬ 
hibit distortion of this type, such as fading channels, amplitude and phase distortion 
typically occur together. For a transmitted sequence of pulses, such distortion 
manifests itself as a signal dispersion or ‘‘smearing" so that any one pulse in the 
received demodulated sequence is not well defined. The overlap or smearing, 
known as intersymhol interference (ISl), described in Section 3.3, arises in most 
modulation systems; it is one of the major obstacles to reliable high-speed data 
transmission over bandlimited channels. In the broad sense, the name “equaliza¬ 
tion" refers to any signal processing or filtering technique that is designed to 
eliminate or reduce ISl. 

In Figure 2.1, equalization is partitioned into two broad categories. The first 
category, maximum-likelihood sequence estimation (MLSE), entails making mea¬ 
surements of /*,,(/) and then providing a means for adjusting the receiver to the 
transmission environment. The goal of such adjustments is to enable the detector 
to make good estimates from the demodulated distorted pulse sequence. With an 
MLSE receiver, the distorted samples are not reshaped or directly compensated in 
any way; instead, the mitigating technique for the MLSE receiver is to adjust itself 
in such a way that it can better deal with the distorted samples. (An example of this 
method, known as Viterbi equalization, is treated in Section 15.7.1.) The second 
category, equalization with filters^ uses filters to compensate the distorted pulses. In 
this second category, the detector is presented with a sequence of demodulated 
samples that the equalizer has modified or “cleaned up" from the effects of ISL 
Equalizing with filters, the more popular approach and the one described in this 
section, lends itself to further partitioning. The filters can be described as to 
whether they are linear devices that contain only feedforward elements transversal 
equalizers), or whether they are nonlinear devices that contain both feedforward 
and feedback elements (decision feedback equalizers). Tliey can be grouped ac¬ 
cording to the automatic nature of their operation, which may be either preset or 
adaptive. They also can be grouped according to the filter's resolution or update 
rate. Are predetection samples provided only on symbol boundaries, that is, one 
sample per symbol? If so, the condition is known as symbol spaced. Are multiple 
samples provided for each symbol? If so, this condition is known as fractionally 
spaced. 

We now modify Equation (3.77) by letting the receiving/equalizing filtef be 
replaced by a separate receiving filter and equalizing filter, defined by frequency 
transfer functions H^(f) and H^(f), respectively. Also, let the overall system trans¬ 
fer function H(f) be a raised-cosine filter, designated //rcCO- Thus, we write 

= H,{f) Hif) m) fiif) (3.83) 

In practical systems, the channel's frequency transfer function H,.(f) and its im¬ 
pulse response h^t) are not known with sufficient precision to allow for a receiver 
design to yield zero ISl for all time. Usually, the transmit and receive filters are 
chosen to be matched so that 


fkeif) = mf) Kif) 


(3.84) 
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In this way, //,(/) and HXf) each have frequency transfer functions that are the 
square root of the raised cosine (root-raised cosine). Then, the equalizer transfer 
function needed to compensate for channel distortion is simply the inverse of the 
channel transfer function: 


W) = 




(3.85) 


Sometimes a system frequency transfer function manifesting ISI at the sam¬ 
pling points is purposely chosen (e.g., a Gaussian filter transfer function). The mo¬ 
tivation for such a transfer function is to improve bandwidth efficiency, compared 
with using a raised-cosine filter. When such a design choice is made, the role of the 
equalizing filter is not only to compensate for the channel-induced ISI, but also to 
compensate for the ISI brought about by the transmitter and receiver filters [7]. 


3.4.2 Eye Pattern 

An eye pattern is the display that results from measuring a system’s response to 
baseband signals in a prescribed way. On the vertical plates of an oscilloscope we 
connect the receiver’s response to a random pulse sequence. On the horizontal 
plates we connect a sawtooth wave at the signaling frequency. In other words, the 
horizontal time base of the oscilloscope is set equal to the symbol (pulse) duration. 
This setup superimposes the waveform in each signaling interval into a family of 
traces in a single interval (0, T). Figure 3.24 illustrates the eye pattern that results 
for binary antipodal (bipolar pulse) signaling. Because the symbols stem from a 
random source, they are sometimes positive and sometimes negative, and the per¬ 
sistence of the cathode ray tube display allows us to see the resulting pattern 
shaped as an eye. The width of the opening indicates the time over which sampling 
for detection might be performed. Of course, the optimum sampling time corre¬ 
sponds to the maximum eye opening, yielding the greatest protection against noise. 


Figure 3.24 Eye Pattern. 
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If there were no filtering in the system—that is, if the bandwidth corresponding to 
the transmission of these data pulses were infinite—then the system response 
would yield ideal rectangular pulse shapes. In that case, the pattern would look like 
a box rather than an eye. In Figure 3.24, the range of amplitude differences labelled 
is a measure of distortion caused by ISI, and the range of lime differences of the 
zero crossings labelled is a measure of the timing jitter. Measures of noise mar¬ 
gin and sensitivity-to-timing error 57 are also shown in the figure. In general, 
the most frequent use of the eye pattern is for qualitatively assessing the extent of 
the ISI. As the eye closes, ISI is increasing; as the eye opens, ISI is decreasing. 


3.4.3 Equalizer Filter Types 

3.4.3.1 Transversal Equalizer 

A training sequence used for equalization is often chosen to be a noise-like 
sequence, “rich” in spectral content, which is needed to estimate the channel fre¬ 
quency response. In the simplest sense, training might consist of sending a single 
narrow pulse (approximately an ideal impulse) and thereby learning the impulse 
response of the channel. In practice, a pseudonoise (PN) signal is preferred over a 
single pulse for the training sequence because the PN signal has larger average 
power and hence larger SNR for the same peak transmitted power. For describing 
the transversal filter, consider that a single pulse was transmitted over a system 
designated to have a raised-cosine transfer function //rc(/) = Also 

consider that the channel induces LSI, so that the received demodulated pulse ex¬ 
hibits distortion, as shown in Figure 3.25, such that the pulse sidelobes do not go 
through zero at sample times adjacent to the mainlobe of the pulse. The distortion 
can be viewed as positive or negative echoes occurring both before and after the 
mainlobe. To achieve the desired raised-cosine transfer function, the equalizing fil¬ 
ter should have a frequency response H^{f ), as shown in Equation (3.85), such that 
the actual channel response when multiplied by H^{f) yields //rc(/)- In other 
words, we would like the equalizing filter to generate a set of canceling echoes. 



Figure 3.25 Received pulse exhibiting distortion. 
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Since we are interesled in sampling the equalized waveform at only a few predeter¬ 
mined sampling times, then the design of an equalizing filter can be a straightfor¬ 
ward task. 

The transversal filter, depicted in Figure 3.26, is the most popular form of an 
easily adjustable equalizing filter consisting of a delay line with T-second taps 
(where T is the symbol duration). In such an equalizer, the current and past values 
of the received signal are linearly weighted with equalizer coefficients or tap 
weights |C;,| and are then summed to produce the output. The main contribution is 
from a central tap, with the other taps contributing echoes of the main signal at 
symbol intervals on either side of the main signal. If it were possible for the filter to 
have an infinite number of taps, then the tap weights could be chosen to force the 
system impulse response to zero at all but one of the sampling times, thus making 
//(.(/) correspond exactly to the inverse of the channel transfer function in Equa¬ 
tion (3.85). Even though an infinite length filter is not realizable, one can still 
specify practical filters that approximate the ideal case. 

In Figure 3.26, the outputs of the weighted taps are amplified, summed, and 
fed to a decision device. The tap weights |c,J need to be chosen so as to subtract the 
effects of interference from symbols adjacent in time to the desired symbol. Con¬ 
sider that there are (2A^ + 1) taps with weights c.v, c*_;v+ii • • • ^ Output samples 
{z(/:)l ^f Ihe equalizer are then found by convolving the input samples {.v(A:)) and 
tap weights lc„) as follows; 

.V 

k = -2N,.,.,2N n = -N,...,N (3.86) 

/i = -V 

where * = 0, ±1, ±2,... is a time index that is shown in parentheses. (Time may lake 
on any range of values.) Fhe index // is used two ways—as a time offset, and as a fil¬ 
ter coefficient identifier (which is an address in the filter). When used in the latter 




Algorithm for 
coefficient adjustment 


yk 


Figure 3.26 Transversal filter. 
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sense, it is shown as a subscript. By defining the vectors z and c and the matrix x as, 
respectively, 
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(3.87) 


(3.88) 


we can describe the relationship among {z(/£:)l, {x(^)), and {c„] more compactly as 


Z = X c (3.89a) 

And whenever the matrix x is square, with its rows and columns each having the 
same dimension as the number of elements in c, we can find c by solving the follow¬ 
ing equation; 


c = X 'z (3.89b) 

Notice that the size of the vector z and the number of rows in the matrix x may be 
chosen to be any value, because one might be interested in the ISI at sample points 
far removed from the mainlobe of the pulse in question. In Equations (3.86) 
through (3.88), the index k was arbitrarily chosen to allow for 4N + 1 sample points. 
The vectors z and c have dimensions 4A^ + 1 and 2N + 1, respectively, and the ma¬ 
trix X is nonsquare with dimensions 4A^ + 1 by 2N + 1. Such equations are referred 
to as an overdetermined set (i.e., there are more equations than unknowns). One 
can solve such a problem in a deterministic way known as the zero-forcing solution, 
or, in a statistical way, known as the minimum mean-square error (MSE) solution. 


Zero-Forcing Solution. This solution starts by disposing of the top N and 
bottom N rows of the matrix \ in Equation (3.88), thereby transforming x into a 
square matrix of dimension 2N -i- 1 by 2N + 1, transforming z into a vector of 
dimension 2N + 1, and yielding in Equation (3.89a), a deterministic set of 2N + 1 
simultaneous equations. This zero-forcing solution minimizes the peak ISI distor¬ 
tion by selecting the |c„l weights so that the equalizer output is forced to zero at N 
sample points on either side of the desired pulse. In other words, the weights are 
chosen so that 
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zik) = 


1 for ^ = 0 
0 for k = ±1, ±2,, ±A^ 


(3.90) 


Equation (3.89) is used to solve the 2N + 1 simultaneous equations for the set of 
2N + 1 weights {c„]. The required length of the filter (number of tap weights) is a 
function of how much smearing the channel may introduce. For such an equalizer 
with finite length, the peak distortion is guaranteed to be minimized only if the eye 
pattern is initially open. However, for high-speed transmission and channels 
introducing much ISI, the eye is often closed before equalization [8]. Since the zero¬ 
forcing equalizer neglects the effect of noise, it is not always the best system solution. 

Example 3.5 A Zero-Forcing Three-Tap Equalizer 

Consider that the tap weights of an equalizing transversal filter are to be determined 
by transmitting a single impulse as a training signal. Let the equalizer circuit in Figure 
3.26 be made up of just three taps. Given a received distorted set of pulse samples 
ljc(it)|, with voltage values 0.0, 0.2, 0.9, -0.3, O.l, as shown in Figure 3.25, use a zero¬ 
forcing solution to find the weights (c.i, c,>, Ci| that reduce the ISI so that the equalized 
pulse samples lz(^)) have the values, |z(-l) = 0, z(0) = 1, z(l) = 0}. Using these 
weights, calculate the ISl values of the equalized pulse at the sample times k=±2,±3. 
What is the largest magnitude sample contributing to ISI, and what is the sum of all 
the ISI magnitudes? 

Solution 

For the channel impulse response specified. Equation (3.89) yields 

Z = X c 

or 

"0] rjr(O) x(~l) x{~2)l[c.-i' 

1 = x{\) j:(0) x{-l) C(i 

.0_ .^(2) x{\) jc(0) _ . Cl _ 

0.9 0.2 

= -0.3 0.9 0.2 Co 

O.l -0.3 O.9J L c, _ 

Solving these three simultaneous equations results in the following weights: 

“c.j] r-0.2140' 

Co = 0.9631 

c, J L 0.3448. 

The values of the equalized pulse samples |z(fc)l corresponding to sample times k = 
-3, -2, -1,0,1, 2,3 are computed by using the preceding weights in Equation (3.89a), 
yielding 


The sample of greatest magnitude contributing to ISI equals 0.0428, and the sum of all 
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The values of the equalized pulse samples (^(A')) corresponding to sample times k = 
-3, -2, -1,0,1,2, 3 are computed by using the preceding weights in Equation (3.89a), 
yielding 

0.0000, -0.0428. 0.0000, 1.0000, O.CXXK), -0.0071, 0.0345 

The sample of greatest magnitude contributing to ISI equals 0.0428, and the sum of all 
the ISI magnitudes equals 0.0844. It should be clear that this three-lap equalizer has 
forced the sample points on either side of the equalized pulse to be zero. If the equal¬ 
izer is made longer than three taps, more of the equalized sample points can be forced 
to a zero value. 

Minimum MSE Solution, A more robust equalizer is obtained if the |c„} 
tap weights are chosen to minimize the mean-square error (MSE) of all the ISI 
terms plus the noise power at the output of the equalizer [9]. MSE is defined as the 
expected value of the squared difference between the desired data symbol and the 
estimated data symbol. One can use the set of overdetermined equations to obtain 
a minimum MSE solution by multiplying both sides of Equation (3.89a) by x^, 
which yields flO] 

x^z = x^xc (3.91a) 

and 

R.C - (3.91b) 

where = x^z is called the cross-correlation vector and = x^x is called the au¬ 
tocorrelation matrix of the input noisy signal. In practice, R^^ and R^^ are unknown 
a priori, but can be approximated by transmitting a test signal over the channel and 
using time average estimates to solve for the tap weights from Equation (3.91), as 
follows: 


c = RJR,, (3.92) 

In the case of the deterministic zero-forcing solution, the x matrix must be square. 
But to achieve the minimum MSE (statistical) solution, one starts with an overde¬ 
termined set of equations and hence a nonsquare x matrix, which then gets trans¬ 
formed to a square autocorrelation matrix R^^ = x^x, yielding a set of 2N + 1 
simultaneous equations, whose solution leads to tap weights that minimize the 
MSE. The size of the vector c and the number of columns of the matrix x corre¬ 
spond to the number of taps in the equalizing filter. Most high-speed telephone¬ 
line modems use an MSE weight criterion because it is superior to a zero-forcing 
criterion; it is more robust in the presence of noise and large ISI [8]. 

Example 3.6 A Minimum MSE 7-Tap Equalizer 

Consider that the tap weights of an equalizing transversal filter are to be determined 
by transmitting a single impulse as a training signal. Let the equalizer circuit in Figure 
3.26 be made up of seven taps. Given a received distorted set of pulse samples {x(A^)), 
with values 0.0108, -0.0558, 0.1617, 1.0000, -0.1749, 0.0227, 0.0110, use a minimum 
MSE solution to find the value of the weights |c„| that will minimize the ISI. With 
these weights, calculate the resulting values of the equalized pulse samples at the fol- 
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lowing times: [k - 0, ±1, ±2,, ±6). What is the largest magnitude sample contribut¬ 
ing to TSI, and what is the sum of all the ISI magnitudes? 

Solution 


For a seven-tap filter (N = 
mensions 4A^ + 1 by 2^ + 1 

3), one can form the x matrix in Equation (3.88) that has 
= 13x7: 


“ 0.0110 

0 
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0 
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0 
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0.0227 
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0 
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0 
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0.0108 

-0.0558 


_ 0 

0 

0 
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0 

0 

0.0108 


Using this x matrix, one can form the autocorrelation matrix R,, and the cross- 
correlation vector defined in Equation (3.91). With the help of a computer to 
invert and perform matrix multiplication, the solution for the tap weights 
(c_ 3 , c_ 2 , c_i, Co, Cl, C2, Cl) showu in Equation (3.92) yields 

-0.0116, 0.0108, 0.1659, 0.9495, -0.1318, 0.0670, -0.0269 

Using these weights in Equation (3.89a), we solve for the 13 equalized samples {z(A')l 
at times k = -6, -5, ...,5,6: 

-0.0001, -O.OOOl, 0.0041, 0.0007, O.OOtXJ, -0.0000, 1.0000, 0.0003, 
-0.0007, 0.0015, -0.0095, 0.0022, -0.0003 

The largest magnitude sample contributing to ISI equals 0.0095, and the sum of all the 
ISI magnitudes equals 0.0195. 

3A3.2 Decision Feedback Equalizer 

The basic limitation of a linear equalizer, such as the transversal filter, is that 
it performs poorly on channels having spectral nulls [11]. Such channels are often 
encountered in mobile radio applications. A decision feedback equalizer (DFE) is 
a nonlinear equalizer that uses previous detector decisions to eliminate the ISI on 
pulses that are currently being demodulated. The ISI being removed was caused by 
the tails of previous pulses; in effect, the distortion on a current pulse that was 
caused by previous pulses is subtracted. 

Figure 3.27 shows a simplified block diagram of a DFE where the forward 
filter and the feedback filter can each be a linear filter, such as a transversal filter. 
The figure also illustrates how the filter tap weights are updated adaptively. (See 
the following section.) The nonlinearity of the DFE stems from the nonlinear 


3.4 Equalization 


157 



Forward Equalized 

filter samples Detector 
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Figure 3.27 Decision Feedback Equalizer. 


characteristic of the detector that provides an input to the feedback filter. The 
basic idea of a DFE is that if the values of the symbols previously detected arc 
known (past decisions are assumed to be correct), then the ISI contributed by these 
symbols can be canceled out exactly at the output of the forward filter by subtract¬ 
ing past symbol values with appropriate weighting. The forward and feedback tap 
weights can be adjusted simultaneously to fulfill a criterion such as minimizing 
the MSE. 

When only a forward filter is used, the output of the filter contains channel 
noise contributed from every sample in the filter. The advantage of a DFE imple¬ 
mentation is that the feedback filter, which is additionally working to remove ISI, 
operates on noiseless quantized levels, and thus its output is free of channel noise. 


3.4.4 Preset and Adaptive Equalization 

On channels whose frequency responses are known and time invariant, the channel 
characteristics can be measured and the filter’s tap weights adjusted accordingly. If 
the weights remain fixed during transmission of data, the equalization is called pre¬ 
set equalization; one very simple method of preset equalization consists of setting 
the tap weights {c„] according to some average knowledge of the channel. This was 
used for data transmission over voice-grade telephone lines at less than 2400 bit/s. 
Another preset method consists of transmitting a training sequence that is com¬ 
pared at the receiver with a locally generated sequence. The differences between 
the two sequences are used to set {c„]. The significant aspect of any preset method 
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is that it is done once at the start of transmission or seldom (when transmission is 
broken and needs to be reestablished). 

Another type of equalization, capable of tracking a slowly lime-varying chan¬ 
nel response, is known as adaptive equalization. It can be implemented to perform 
tap-weight adjustments periodically or continually. Periodic adjustments are ac¬ 
complished by periodically transmitting a preamble or short training sequence of 
digital data that is known in advance by the receiver. The receiver also uses the 
preamble to detect start of transmission, to set the automatic gain control (AGC) 
level, and to align internal clocks and local oscillator with the received signal. Con¬ 
tinual adjustments are accomplished by replacing the known training sequence 
with a sequence of data symbols estimated from the equalizer output and treated as 
known data. When performed continually and automatically in this way, the adap¬ 
tive procedure (the most popular) is referred to as decision directed [11]. The name 
“decision directed’' is not to be confused with decision feedback (DFE). Decision 
directed only addresses how filter tap weights are adjusted—that is, with the help 
of a signal from the detector. DFE, however, refers to the fact that there exists an 
additional filter that operates on the detector output and recursively feeds back a 
signal to the detector input. Thus, with DFE there are two filters, a feed-forward 
filter and a feedback filter that process the data and help mitigate the ISl. 

A disadvantage of preset equalization is that it requires an initial training 
period that must be invoked at the start of any new transmission. Also, a time- 
varying channel can degrade system performance due to ISI, since the tap 
weights are fixed. Adaptive equalization, particularly decision-directed adaptive 
equalization, successfully cancels ISI when the initial probability of error due to 
probability of error exceeds one percent, (rule of thumb). If the probability of 
error exceeds one percent, the decision directed equalizer might not converge. A 
common solution to this problem is to initialize the equalizer with an alternate 
process, such as a preamble to provide good channel-error performance, and then 
switch to the decision-directed mode. To avoid the overhead represented by a 
preamble, many systems designed to operate in a continuous broadcast mode use 
blind equalization algorithms to form initial channel estimates. These algorithms 
adjust filter coefficients in response to sample statistics rather than in response to 
sample decisions [11]. 

Automatic equalizers use iterative techniques to estimate the optimum coeffi¬ 
cients. The simultaneous equations described in Equation (3.89) do not include the 
affects of channel noise. To obtain a stable solution to the filter weights, it is neces¬ 
sary that the data be averaged to obtain stable signal statistics, or the noisy solu¬ 
tions obtained from the noisy data must be averaged. Considerations of algorithm 
complexity and numerical stability most often lead to algorithms that average noisy 
solutions. The most robust of this class of algorithm is the least-mean-square 
(LMS) algorithm. Each iteration of this algorithm uses a noisy estimate of the error 
gradient to adjust the weights in the direction to reduce the average mean-square 
error. The noisy gradient is simply the product e(/c) of an error scalar e{k) and 
the data vector r^. The vector is the vector of noise-corrupted channel samples 
residing in the equalizer filter at time k. Earlier, an impulse was transmitted and 
the equalizing filter operated on a sequence of samples (a vector) that represented 
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the impulse response of the channel. We displayed these received samples (in time- 
shifted fashion) as the matrix x. Now, rather than dealing with the response to an 
impulse, consider that data is sent and thus the vector of received samples at the 
input to the filter (Figure 3.27) represents the data response of the channel. The 
error is formed as the difference between the desired output signal and the filter 
output signal and is given by 


e{k) = z{k) - i{k) (3.93) 

where z{k) is the desired output signal (a sample free of ISI) and z(A:) is an 
estimate of z{k) at time k out of the filter (into the quantizer of Figure 3.27), which 
is obtained as follows: 


zik) = c\= 2 (3-94) 

n = —N 

In Equation (3.94), the summation represents a convolution of the input data 
samples with the |c„l tap weights, where c„ refers to the «th tap weight at time k. 
and is the transpose of the weight vector at time k. We next show the iterative 
process that updates the set of weights at each lime k as 

c{k + 1) = c(^) + Ae{k)T, (3.95) 

where c{k) is the vector of filter weights at time k, and A is a small term that limits 
the coefficient step size and thus controls the rate of convergence of the algorithm 
as well as the variance of the steady stale solution. This simple relationship is a con¬ 
sequence of the orthogonality principle that slates that the error formed by an opti¬ 
mal solution is orthogonal to the processed data. Since this is a recursive algorithm 
(in the weights), care must be exercised to assure algorithm stability. Stability is as¬ 
sured if the parameter A is smaller than the reciprocal of the energy of the data in 
the filter. When stable, this algorithm converges in the mean to the optimal solu¬ 
tion but exhibits a variance proportional to the parameter A. Thus, while we want 
the convergence parameter A to be large for fast convergence but not so large as to 
be unstable, we also want it to be small enough for low variance. The parameter A 
is usually set to a fixed small amount [12] to obtain a low-variance steady-state lap- 
weight solution. Schemes exist that permit A to change from large values during 
initial acquisition to small values for stable steady-state solutions [13]. 

Note that Equations (3.93) through (3.95) are shown in the context of real sig¬ 
nals. When the receiver is implemented in quadrature fashion, such that the signals 
appear as real and imaginary (or inphase and quadrature) ordered pairs, then each 
line in Figure 3.27 actually consists of two lines, and Equations (3.93) through 
(3.95) need to be expressed with complex notation. (Such quadrature implementa¬ 
tion is treated in greater detail in Sections 4.2.1 and 4.6.) 
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3.4.5 Filter Update Rate 


Equalizer filters are classified by the rate at which the input signal is sampled, A 
transversal filter with taps spaced T seconds apart, where T is the symbol lime, is 
called a symbol-spaced equalizer. The process of sampling the equalizer output at a 
rate 1/7 causes aliasing if the signal is not strictly bandlimited to 1/7 hertz—that is, 
the signal’s spectral components spaced 1/7 hertz apart are folded over and super¬ 
imposed. The aliased version of the signal may exhibit spectral nulls [8]. A filter up¬ 
date rate that is greater than the symbol rate helps to mitigate this difficulty. 
Equalizers using this technique are called fractionally-spaced equalizers. With a 
fractionally spaced equalizer, the filter taps arc spaced at 



(3.96) 


seconds apart, where r denotes the excess bandwidth. In other words, the received 
signal bandwidth is 



(3.97) 


The goal is to choose 7' so that the equalizer transfer function //,.(/) becomes suffi¬ 
ciently broad to accommodate the whole signal spectrum. Note that the signal at 
the output of the equalizer is still sampled at a rate 1/7, but since the lap weights 
are spaced 7' seconds apart (the equalizer input signal is sampled at a rate 7/7'), 
the equalization action operates on the received signal before its frequency compo¬ 
nents are aliased. Equalizer simulations over voice-grade telephone lines, with 7' = 
7/2, confirm that such fractionally-spaced equalizers outperform symbol-spaced 
equalizers [14]. 


3.5 CONCLUSION 

In this chapter, we described the detection of binary signals plus Gaussian noise in 
terms of two basic steps. In the first step the received waveform is reduced to a sin¬ 
gle number z(7), and in the second step a decision is made as to which signal was 
transmitted, on the basis of comparing z{T) to a threshold. We discussed how to 
best choose this threshold. We also showed that a linear filter known as a matched 
filter or correlator is the optimum choice for maximizing the output signal-to-noise 
ratio and thus minimizing the probability of error. 

We defined intersymbol interference (ISI) and explained the importance of 
Nyquist’s work in establishing a theoretical minimum bandwidth for symbol detec¬ 
tion without ISI. We partitioned error-performance degradation into two main 
types. The first is a simple loss in signal-to-noise ratio. The second, resulting from 
distortion, is a bottoming-out of the error probability versus the curve. 
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Finally, we described equalization techniques that can be used to mitigate the 
effects of ISI. 
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PROBLEMS 


3.1. Determine whether or not Si(t) and ^2(0 orthogonal over the interval (-1.57'2 
<t< 1 . 572 ), where j|(r) = cos {luft + 52(/) = cos ( 217 / 2 / + ^ 2 )^ ^i^d /2 = 1/72 ^he 

following cases. 

(a) /i=/ 2 and<|>i = <|)2 

(h) /i = 3/2 and <|), = (|)2 
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Vi(^) V3(<) 



Figure P3.2 


(c) /i = 2 / 2 andd)i=(l )2 

(d) /| = 77/2 and 4>i =<t >2 

(c) fi = fi and 4)| = 4)2 + 77/2 
(f) /i = fi and (|>i = 4>2 + '17 

3.2. (a) Show that the three functions illustrated in Figure P3.1 are pairwise orthogonal 

over the interval (- 2 , 2 ). 

(b) Determine the value of the constant. A, that makes the set of functions in part 
(a) an orthonormal set. 

(c) Express the following waveform, x(r), in terms of the orthonormal set of part (b). 

^ ^ 11 for 0 < r < 2 

\0 otherwise 

3.3. Consider the functions 


4 (i{/) = exp (—|r I) and 4/2 = 1 — >1 exp (—2 |/ 1 ) 


Determine the constant. A, such that and 4 ^ 2(0 are orthogonal over the interval 
(-=»> “). 

3.4. Assume that in a binary digital communication system, the signal component out of 
the correlator receiver is a,{7^ = +1 or -1 V with equal probability. If the Gaussian 
noise at the correlator output has unit variance, find the probability of a bit error. 

3.5. A bipolar binary signal, is a +1- or -1-V pulse during the interval (0, T). Addi¬ 
tive white Gaussian noise having two-sided power spectral density of 10'^ W/Hz is 
added to the signal. If the received signal is detected with a matched filler, determine 
the maximum bit rate that can be sent with a bit error probability of < 10"^ 

3.6. Bipolar pulse signals, (/ = 1.2), of amplitude ±1 V are received in the presence of 
AWGN that has a variance of 0.1 V^. Determine the optimum (minimum probability 
of error) detection threshold, -y(,» for matched filler detection if the a priori probabili¬ 
ties are: (a) P{si) = 0.5; (b) P(^i) = 0.7; (c) P{si) = 0.2. (d) Explain the effect of the 
a priori probabilities on the value of 70 * Refer to Equations (B.IO) to (B,12).l 

3.7. A binary communication system transmits signals 5,(0 (/ = 1, 2). The receiver test 

statistic ziT) = + «(,, where the signal component a, is either ai = +1 or ^2 = -1 and 

the noise component no is uniformly distributed, yielding the conditional density 
functions p(zl5,) given by 



for -0.2 < z < 1.8 
otherwise 


and 
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for — l.S^z ^ 0.2 
otherwise 


Find the probability of a bit error, Pg, for the case of equally likely signaling and the 
use of an optimum decision threshold. 

3.8. (a) What is the theoretical minimum system bandwidth needed for a 10-Mbits/s 

signal using 16-levcl PAM without ISI? 

<b) How large can the filler roll-off factor be if the allowable system bandwidth is 
1.375 MHz? 

3.9. A voice signal (300 to 3300 Hz) is digitized such that the quantization distortion 
< ±0.1% of the peak-to-peak signal voltage. Assume a sampling rate of 8000 sam¬ 
ples/s and a multilevel PAM waveform with M = 32 levels. Find the theoretical mini¬ 
mum system bandwidth that avoids ISI. 

3.10. Binary data at 9600 bits/s are transmitted using 8-ary PAM modulation with a system 
using a raised cosine roll-off filter characteristic. The system has a frequency re¬ 
sponse out to 2.4 kHz. 

(a) What is the symbol rate? 

(b) What is the roll-off factor of the filter characteristic? 

3.11. A voice signal in the range 300 to 3300 Hz is sampled at 8000 samples/s. We may 
transmit these samples directly as PAM pulses or we may first convert each sample 
to a PCM format and use binary (PCM) waveforms for transmission. 

(a) What is the minimum system bandwidth required for the detection of PAM with 
no ISI and with a filter roll-off characteristic of r = 1 ? 

(b) Using the same filter roll-off characteristic, what is the minimum bandwidth re¬ 
quired for the detection of binary (PCM) waveforms if the samples are quantized 
to eight levels? 

(c) Repeat part (b) using 128 quantization levels. 

3.12. An analog signal is PCM formatted and transmitted using binary waveforms over a 
channel that is bandlimitcd to 100 kHz. Assume that 32 quantization levels are used 
and that the overall equivalent transfer function is of the raised cosine type with roll¬ 
off r = 0.6. 

(a) Find the maximum bit rate that can be used by this system without introducing 
ISI. 

(b) Find the maximum bandwidth of the original analog signal that can be accommo¬ 
dated with these parameters. 

(c) Repeat parts (a) and (b) for transmission with 8-ary PAM waveforms. 

3.13. Assume that equally-likely RZ binary pulses are coherently detected over a Gaussian 
channel with A„= 10"^ Watt/Hz. Assume that synchronization is perfect, and that the 
received pulses have an amplitude of 100 mV. If the bit-error probability specification 
hPfi=\0 '-\ find the largest data rate that can be transmitted using this system. 

3.14. Consider that NRZ binary pulses arc transmitted along a cable that attenuates the 
signal power by 3 dB (from transmitter to receiver). The pulses are coherently de¬ 
tected at the receiver, and the data rate is 56 kbit/s. Assume Gaussian noise with 

= lO"*" Watt/Hz. What is the minimum amount of power needed at the transmitter 
in order to maintain a bit-error probability of P^ = 10”^? 

3.15. Show that the Nyquist minimum bandwidth for a random binary sequence sent with 
ideal-shaped bipolar pulses is the same as the noise equivalent bandwidth. Hint: the 
power spectral density for a random bipolar sequence is given in Equation (1.38) and 
the noise equivalent bandwidth is defined in Section 1.7.2. 
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3.16. Consider the 4-ary PAM-modulated sequence of message symbols |+1 +1 -1 -i-3 +1 

+3), where the members of the alphabet set are: (±1, ±3|. The pulses have been 
shaped with a root-raised cosine filter such that the support time of each filtered 
pulse is 6-symbol times, and the transmitted sequence is the analog waveform shown 
in Figure 2.47a. Note that the waveform appears “smeared" due to the filter-induced 
ISI. Show how a bank of N correlators can be implemented to perform matched- 
filler demodulation of the received pulse sequence, r(/), where N corresponds to the 
number of symbols in the pulse-support lime, [Hint: For the bank of correlators, use 
reference signals of the form Si (t - kT), where k = _5 and T is the symbol lime.J 

3.17. A desired impulse response of a communication system is the ideal h{() = 8(r), where 
8(0 is the impulse function. Assume that the channel introduces ISI so that the over¬ 
all impulse response becomes h(t) - 8(0 + a8(/- T), where a < 1, and Fis the symbol 
lime. Derive an expression for the impulse response of a zero-forcing filter that will 
equalize the effects of ISF Demonstrate that this filter suppresses the ISI. If the 
resulting suppression is deemed inadequate, how can the filter design be modified to 
increase the ISI suppression further? 

3.18. The result of a single pulse (impulse) transmission is a received sequence of samples 
(impulse response), with values 0.1, 0.3, -0.2, 1.0, 0.4, -0.1, 0.1, where the leftmost 
sample is the earliest. The value 1.0 corresponds to the mainlobe of the pulse, and 
the other entries correspond to adjacent samples. Design a 3-tap transversal equal¬ 
izer that forces the ISI to be zero at one sampling point on each side of the mainlobe. 

Calculate the values of the equalized output pulses at limes ^: = 0, ±1.±3. After 

equalization, what is the largest magnitude sample contributing to ISI, and what is 
the sum of all the ISI magnitudes? 

3.19. Repeal problem 3.18 for the case of a channel impulse response described by the fol¬ 
lowing received samples: 0.01, 0.02, -0.03, 0.1, 1.0, 0.2, -0.1, 0.05, 0.02. Use a com¬ 
puter to find the weights of a nine-tap transversal equalizer to meet the minimum 
MSE criterion. Calculate the values of the equalized output pulses at times A: = 0, 

±1.±8. After equalization, what is the largest magnitude sample contributing to 

ISI, and what is the sum of all the ISI magnitudes? 

3.20. In this chapter, it has been emphasized that signal-processing devices, such as 
multipliers and integrators, typically deal with signals having units of volts. There¬ 
fore, the transfer functions of such processors must accommodate these units. Draw 
a block diagram of a product-integrator showing the signal units on each of 
the wires, and the device transfer functions in each of the blocks (Hint: sec Sec¬ 
tion 3.2.5.1). 


QUESTIONS 

3.1. In the case of baseband signaling, the received waveforms are already in a pulse-like 
form. Why then, is a demodulator needed to recover the pulse waveform? (See 
Chapter 3, introduction.) 

3.2. Why is a natural figure-of-merit for digital communication systems? (See 

Section 3.1.5.) 

3J. When representing limed events, what dilemma can easily result in confusing the 
most-significant bit (MSB) and the least-significant bit (LSB)? (See Section 3.2.3.1.) 
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3.4. The term matched-filter is often used synonymously with correlator. How is that 
possible when their mathematical operations are different? (See Section 3.2.3.I.) 

3.5. Describe the two fair ways of comparing different curves that depict bit-error proba¬ 
bility versus EfJN^. (See Section 3.2.5.3.) 

3.6. Are there other pulse-shaping filter functions, besides the raised-cosine, that exhibit 
zero ISl? (See Section 3.3.) 

3.7. Describe a reasonable goal in endeavoring to compress bandwidth to the minimum 
possible, without incurring ISl. (See Section 3.3.1.1.) 

3.8. The error performance of digital signaling suffers primarily from two degradation 
types; loss in signal-to-noise ratio, and distortion resulting in an irreducible bit-error 
probability. How do they differ? (See Section 3.3.2.) 

3.9. Often times, providing more will not mitigate the degradation due to intersym¬ 
bol interference (ISl). Explain why this is the case. (See Section 3.3.2.) 

3.10. Describe the difference between equalizers that use a zero-forcing solution, and 
those that use a minimum mean-square error solution? (See Section 3.4.3.1.) 


EXERCISES 

Using the Companion CD, run the exercises associated with Chapter 3. 
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